LlamaForCasualLM: base_model Attribute Mystery
Author: Maxwilson1101
Pull request: no
Labels: name: bug

### System Info

I'm working on a demo to study `pipeline` construction.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

# let's rock the smallest model on hf
MODEL_NAME = "arnir0/Tiny-LLM"

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModelForCausalLM.from_pretrained(MODEL_NAME)

INPUT = ["the secret to baking a really good cake is ", "a baguette is "]

if __name__ == "__main__":
	pipeline = pipeline(task="text-generation", model=MODEL_NAME, device="mps")
	for out in pipeline(INPUT):
		print(out[0]["generated_text"])
```

I find a tricky problem about `base_model` attribute in LlamaForCasualLM.

```python
>>> from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
>>> MODEL_NAME = "arnir0/Tiny-LLM"
>>> tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
>>> model = AutoModelForCausalLM.from_pretrained(MODEL_NAME)
>>> INPUT = ["the secret to baking a really good cake is ", "a baguette is "]
>>> pipeline = pipeline(task="text-generation", model=MODEL_NAME, device="mps")
Device set to use mps
>>> model = pipeline.model
>>> model.base_model
LlamaModel(
  (embed_tokens): Embedding(32000, 192)
  (layers): ModuleList(
    (0): LlamaDecoderLayer(
      (self_attn): LlamaAttention(
        (q_proj): Linear(in_features=192, out_features=192, bias=False)
        (k_proj): Linear(in_features=192, out_features=96, bias=False)
        (v_proj): Linear(in_features=192, out_features=96, bias=False)
        (o_proj): Linear(in_features=192, out_features=192, bias=False)
      )
      (mlp): LlamaMLP(
        (gate_proj): Linear(in_features=192, out_features=1024, bias=False)
        (up_proj): Linear(in_features=192, out_features=1024, bias=False)
        (down_proj): Linear(in_features=1024, out_features=192, bias=False)
        (act_fn): SiLUActivation()
      )
      (input_layernorm): LlamaRMSNorm((192,), eps=1e-05)
      (post_attention_layernorm): LlamaRMSNorm((192,), eps=1e-05)
    )
  )
  (norm): LlamaRMSNorm((192,), eps=1e-05)
  (rotary_emb): LlamaRotaryEmbedding()
)
>>> model.base_model.base_model
LlamaModel(
  (embed_tokens): Embedding(32000, 192)
  (layers): ModuleList(
    (0): LlamaDecoderLayer(
      (self_attn): LlamaAttention(
        (q_proj): Linear(in_features=192, out_features=192, bias=False)
        (k_proj): Linear(in_features=192, out_features=96, bias=False)
        (v_proj): Linear(in_features=192, out_features=96, bias=False)
        (o_proj): Linear(in_features=192, out_features=192, bias=False)
      )
      (mlp): LlamaMLP(
        (gate_proj): Linear(in_features=192, out_features=1024, bias=False)
        (up_proj): Linear(in_features=192, out_features=1024, bias=False)
        (down_proj): Linear(in_features=1024, out_features=192, bias=False)
        (act_fn): SiLUActivation()
      )
      (input_layernorm): LlamaRMSNorm((192,), eps=1e-05)
      (post_attention_layernorm): LlamaRMSNorm((192,), eps=1e-05)
    )
  )
  (norm): LlamaRMSNorm((192,), eps=1e-05)
  (rotary_emb): LlamaRotaryEmbedding()
)
```

The sequence of model.base_model.base_model appears to be recursive without termination. I'm not sure whether it is a design choice or a bug that there is an attribute pointing to LlamaModel itself.

**System Info**

```bash
❯ transformers env

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.57.1
- Platform: macOS-15.6.1-arm64-arm-64bit-Mach-O
- Python version: 3.13.5
- Huggingface_hub version: 0.36.0
- Safetensors version: 0.6.2
- Accelerate version: not installed
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.9.1 (NA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
```

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. copy the code
2. debug it in vscode

### Expected behavior

endless chain of `model.base_model.base_model`

Comments:
# This patch prevents further recursive .base_model attribute nesting in LlamaForCausalLM.
# Only returns an actual base or None, not itself.


I could work on this problem, if it should be treated as a bug

Hi, this is intended behaviour! The property `model.base_model` points to either:

1) The model "stem" without output heads, when the model is a derived model with an output head like `LlamaForCausalLM`
2) The model itself, when the model is already a base model

The code for it is quite simple:

```python
@property
def base_model(self) -> nn.Module:
    """
    `torch.nn.Module`: The main body of the model.
    """
    return getattr(self, self.base_model_prefix, self)
```