INDRA 4B Base
Integrated Neural Dual-Attention Reasoning Architecture
INDRA 4B Base is an experimental INDRA language model from the development of the INDRA architecture by Shashwat Mudgal.
This model represents an early INDRA1 architecture at approximately 4 billion parameters.
The checkpoint was developed as an architecture and language-model research experiment and should be understood as a historical generation of INDRA rather than as the current INDRA architecture.
Status: Historical experimental research checkpoint
Model Details
| Property | Value |
|---|---|
| Model | INDRA 4B Base |
| Architecture | INDRA1 |
| Parameters | ~4.02B |
| Model type | Causal language model |
| Developer | Shashwat Mudgal |
| Status | Historical experimental checkpoint |
| Primary purpose | Architecture research |
About INDRA
INDRA — Integrated Neural Dual-Attention Reasoning Architecture is an architecture-first language-model research project developed by Shashwat Mudgal.
The project investigates alternative language-model architectures with an emphasis on reasoning, structured generation, long-context processing, and architectural experimentation.
INDRA is developed as an independent research effort.
INDRA1
This model represents an early generation of the INDRA architecture, referred to as INDRA1 within the development history of the project.
INDRA1 was an experimental architecture developed to investigate alternative approaches to the internal computation of a causal language model.
This 4B checkpoint represents a larger-scale INDRA1 experiment.
The architecture and implementation in this repository should not be assumed to be identical to later INDRA generations.
Initialization
This model was developed using pretrained Qwen weights as an initialization point.
The use of an upstream pretrained model as initialization should not be interpreted as making this an unmodified Qwen model.
The resulting checkpoint represents an experimental INDRA architecture and subsequent training work.
For the exact upstream model and applicable licensing terms, refer to the corresponding upstream Qwen model documentation and license.
Architectural Research
INDRA1 was developed as an independent architectural research direction.
The repository contains custom INDRA architecture implementation code required to instantiate the model.
The specific implementation represented by this checkpoint is preserved as part of the historical development of INDRA.
Later generations of INDRA may use substantially different architectural mechanisms and should not be assumed to inherit every component of INDRA1.
Intended Use
This model is intended for:
- Research into language-model architectures
- Study of the INDRA1 architecture
- Experimental reasoning research
- Historical analysis of INDRA development
- Language-model experimentation
- Reproduction and analysis of the released checkpoint
Out-of-Scope Use
This model is not intended to be treated as:
- A production conversational assistant
- A safety-critical AI system
- A medical, legal, or financial advisor
- An autonomous decision-making system
- A state-of-the-art reasoning model
- The current INDRA architecture
This is a historical research checkpoint.
Capabilities
INDRA 4B Base is an autoregressive causal language model capable of generating text from textual context.
Its capabilities depend on the INDRA1 architecture, the upstream initialization, subsequent training, and the particular checkpoint represented by this repository.
The model should not be assumed to possess reliable reasoning capabilities solely from its architectural design or parameter count.
Limitations
As an experimental research checkpoint, the model may exhibit:
- Hallucination
- Incorrect reasoning
- Repetition
- Inconsistent generation
- Poor instruction following
- Factual inaccuracies
- Prompt sensitivity
- Context-related limitations
- Training artifacts
- Unexpected behavior resulting from experimental architecture and training
No claim of state-of-the-art performance is made unless supported by documented evaluation results.
Historical Context
This repository is preserved as part of the development history of INDRA.
The model represents an early 4B-scale implementation of the INDRA1 architecture.
It should not be interpreted as the final or current INDRA model.
Its purpose includes documenting the progression of the INDRA research program from early architectural experiments toward subsequent generations.
Relationship to Later INDRA Models
INDRA has undergone multiple architectural and training iterations.
This INDRA1 checkpoint predates later generations including INDRA2, INDRA3, and INDRA-Lite.
Later models may differ substantially in:
- Architecture
- Attention mechanisms
- Tokenization
- Initialization
- Training procedure
- Model scale
- Memory mechanisms
- Routing
- Long-context implementation
The existence of this checkpoint does not imply that later INDRA models use the same implementation.
Reproducibility
When conducting research using this checkpoint, record:
- Exact repository revision
- Model checkpoint
- Configuration
- Tokenizer
- Generation parameters
- Transformers version
- PyTorch version
- Hardware
- Training configuration
Results from later INDRA generations should not automatically be considered comparable with this model.
Safety Considerations
This is an experimental language model and has not been presented as a fully safety-aligned conversational system.
It may generate inaccurate, biased, offensive, or otherwise undesirable content.
Appropriate safeguards should be implemented before using the model in applications.
Licensing and Third-Party Components
This model incorporates pretrained upstream model weights as its initialization.
Users must comply with the applicable license governing the upstream Qwen model as well as the license governing this repository and its original INDRA components.
Users should review all applicable licensing terms before redistributing, modifying, or commercially deploying the model.
The INDRA architecture and original implementation are independent research work by Shashwat Mudgal.
The use of upstream initialization weights does not transfer ownership of the INDRA architecture to the upstream model authors.
This section is not legal advice.
Attribution
INDRA
INDRA — Integrated Neural Dual-Attention Reasoning Architecture
Developed by:
Shashwat Mudgal
Upstream Initialization
This model was developed using pretrained Qwen weights as an initialization point.
The applicable Qwen model should be acknowledged according to its upstream license and documentation.
Citation
If you use this checkpoint in research or derivative work, please identify the exact checkpoint and acknowledge the INDRA project.
Suggested citation:
Mudgal, Shashwat. INDRA 4B Base:
An INDRA1 Architecture Research Checkpoint. 2026.
Historical Status
This repository is intentionally preserved as a historical INDRA artifact.
It represents an earlier stage of the architecture's development and should not be interpreted as the current INDRA implementation.
The repository provides a record of an early 4B-scale INDRA architecture experiment.
Disclaimer
INDRA is an independent research project developed by Shashwat Mudgal.
This model incorporates pretrained upstream weights for initialization.
The use of those weights does not imply affiliation with or endorsement by the upstream model authors.
INDRA1 and subsequent INDRA architectures are research projects and should be evaluated independently before deployment.
- Downloads last month
- 15