Hugging Face's logo

Evaluate documentation

Hub methods

You are viewing main version, which requires installation from source. If you'd like regular pip install, checkout the latest stable version (v0.4.0).
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Hub methods

Methods for using the Hugging Face Hub:

Push to hub

evaluate.push_to_hub

< >

( model_id: str task_type: str dataset_type: str dataset_name: str metric_type: str metric_name: str metric_value: float task_name: str = None dataset_config: str = None dataset_split: str = None dataset_revision: str = None dataset_args: Dict = None metric_config: str = None metric_args: Dict = None overwrite: bool = False )

Parameters

  • model_id (str) — Model id from https://hf.co/models.
  • task_type (str) — Task id, refer to the Hub allowed tasks for allowed values.
  • dataset_type (str) — Dataset id from https://hf.co/datasets.
  • dataset_name (str) — Pretty name for the dataset.
  • metric_type (str) — Metric id from https://hf.co/metrics.
  • metric_name (str) — Pretty name for the metric.
  • metric_value (float) — Computed metric value.
  • task_name (str, optional) — Pretty name for the task.
  • dataset_config (str, optional) — Dataset configuration used in load_dataset. See load_dataset for more info.
  • dataset_split (str, optional) — Name of split used for metric computation.
  • dataset_revision (str, optional) — Git hash for the specific version of the dataset.
  • dataset_args (dict[str, int], optional) — Additional arguments passed to load_dataset.
  • metric_config (str, optional) — Configuration for the metric (e.g. the GLUE metric has a configuration for each subset).
  • metric_args (dict[str, int], optional) — Arguments passed during compute().
  • overwrite (bool, optional, defaults to False) — If set to True an existing metric field can be overwritten, otherwise attempting to overwrite any existing fields will cause an error.

Pushes the result of a metric to the metadata of a model repository in the Hub.

Example:

>>> push_to_hub(
...     model_id="huggingface/gpt2-wikitext2",
...     metric_value=0.5
...     metric_type="bleu",
...     metric_name="BLEU",
...     dataset_name="WikiText",
...     dataset_type="wikitext",
...     dataset_split="test",
...     task_type="text-generation",
...     task_name="Text Generation"
... )
< > Update on GitHub

←Saving methods Evaluator classes→