docs/Training_Orchestrator.md

graph LR
    Training_Orchestrator["Training Orchestrator"]
    Model_Abstraction["Model Abstraction"]
    Data_Pipeline["Data Pipeline"]
    Configuration["Configuration"]
    Training_Orchestrator -- "Uses" --> Model_Abstraction
    Training_Orchestrator -- "Consumes" --> Data_Pipeline
    Training_Orchestrator -- "Configured by" --> Configuration
    click Training_Orchestrator href "https://github.com/Josephrp/SmolFactory/blob/main/SmolFactory/docs/Training_Orchestrator.md" "Details"
    click Model_Abstraction href "https://github.com/Josephrp/SmolFactory/blob/main/SmolFactory/docs/Model_Abstraction.md" "Details"
    click Data_Pipeline href "https://github.com/Josephrp/SmolFactory/blob/main/SmolFactory/docs/Data_Pipeline.md" "Details"

Details

One paragraph explaining the functionality which is represented by this graph. What the main flow is and what is its purpose.

Training Orchestrator [Expand]

Implements the core training and fine-tuning loop. This includes managing forward and backward passes, optimization, loss calculation, and integration with acceleration libraries (e.g., accelerate). It also handles callbacks and evaluation logic.

Related Classes/Methods:

• src.trainer (1:9999)

Model Abstraction [Expand]

Provides an abstract interface for loading, configuring, and managing different language models. It handles model initialization, tokenizer loading, and potentially quantization settings, ensuring compatibility with various model architectures and training setups.

Related Classes/Methods:

• src.model (1:9999)

Data Pipeline [Expand]

Manages the entire data processing workflow, from loading raw datasets to tokenization, formatting, and preparing data for training. It ensures efficient data handling, including features like dataset sharding, shuffling, and batching.

Related Classes/Methods:

• src.data (1:9999)

Configuration

Centralizes all configurable parameters for the training process, including model parameters, training arguments, dataset paths, and optimization settings. It provides a structured way to define and access these settings, enabling easy modification and experimentation.

Related Classes/Methods:

• src.config (1:9999)

FAQ