beaglabs/ent-smolm-entity-v2

.ent SmolLM Entity v2

.ent SmolLM Entity v2 is an experimental entity-conditioned language model built on top of HuggingFaceTB/SmolLM2-1.7B-Instruct.

This repository does not contain a full standalone Hugging Face model checkpoint. It contains the pieces needed by the .ent architecture:

• a LoRA adapter for the SmolLM2 base model
• an entity_proj.safetensors projection layer
• an entity_decoder.safetensors entity embedding/structure module

The intended load path is the .ent wrapper in this project, which reconstructs the full model by combining:

1. HuggingFaceTB/SmolLM2-1.7B-Instruct
2. the LoRA adapter in lora/
3. the entity decoder
4. the entity projection layer

What This Model Is

The model augments a standard causal LM with a parallel entity stream:

• tokens are mapped to hashed entity IDs
• entity IDs are decoded into learned entity embeddings
• entity embeddings are projected into the LM hidden space
• projected entity features are added to token embeddings before generation

In the current .ent system, this model is used as one component inside a broader inference engine that includes:

• abstraction layers
• graph-based reasoning
• working memory
• durable semantic/procedural memory
• simple program-like execution for arithmetic/logic tasks

Files

• lora/adapter_model.safetensors: LoRA weights for SmolLM2
• lora/adapter_config.json: PEFT adapter configuration
• entity_proj.safetensors: learned entity-to-hidden projection
• entity_decoder.safetensors: entity embedding/decoder module

Intended Use

This checkpoint is intended for:

• experiments with entity-conditioned generation
• .ent inference and evaluation
• research on structured inference over hashed entities

It is not intended as a drop-in replacement for a normal text-generation model unless you also use the .ent loading code.

Loading

Project-native loading

from transformers import AutoTokenizer
from ent.training.train import EntitySmolWrapper

base_model = "HuggingFaceTB/SmolLM2-1.7B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(base_model)
tokenizer.pad_token = tokenizer.eos_token

model = EntitySmolWrapper.from_pretrained(
    path="architecture/smolm-entity-v2",
    base_model_name=base_model,
    device="cpu",
    tokenizer=tokenizer,
)

lm_eval / .ent evaluation

This repository is also used through the .ent evaluation wrapper:

modal run ent/training/eval.py --model-path /data/output/smolm-entity-v2/final --tag ent-v2

Training Summary

This checkpoint was produced by fine-tuning SmolLM2 with:

• frozen entity decoder
• learned entity projection layer
• LoRA adapters on the language model
• entity-conditioned generation through hashed token/entity features

The training code lives in:

• ent/training/train.py
• ent/training/modal_train.py

Limitations

• This is an experimental checkpoint.
• It depends on external .ent loading code.
• It is not a fully packaged standalone Transformers model repository.
• The broader .ent system is still evolving, especially its inference, memory, and graph-reasoning components.

Evaluation Context

This model has been evaluated inside the .ent inference stack rather than only as a raw decoder. The broader project is moving toward an explicit inference architecture instead of relying purely on single-pass generation.

Citation

If you use this checkpoint, cite the repository/project that introduced the .ent architecture and this entity-conditioned SmolLM variant.