Lighteval documentation
Pipeline
Pipeline
Pipeline
class lighteval.pipeline.Pipeline
< source >( tasks: str pipeline_parameters: PipelineParameters evaluation_tracker: EvaluationTracker model_config: lighteval.models.abstract_model.ModelConfig | None = None model = None metric_options = None )
PipelineParameters
class lighteval.pipeline.PipelineParameters
< source >( launcher_type: ParallelismManager job_id: int = 0 dataset_loading_processes: int = 1 nanotron_checkpoint_path: str | None = None custom_tasks_directory: str | None = None num_fewshot_seeds: int = 1 max_samples: int | None = None cot_prompt: str | None = None remove_reasoning_tags: bool = True reasoning_tags: str | list[tuple[str, str]] = "[('<think>', '</think>')]" load_responses_from_details_date_id: str | None = None bootstrap_iters: int = 1000 )
ParallelismManager
class lighteval.pipeline.ParallelismManager
< source >( value names = None module = None qualname = None type = None start = 1 )
An enumeration.