Title: same value of precision, recall, f1 score at each epoch for classification task. 
Author: Dhanachandra (NONE)
State: closed
Created: 2022-01-27T10:14:16Z
Labels: bug

**1st Epoch:** 
1/27/2022 09:30:48 - INFO - datasets.metric - Removing /home/ubuntu/.cache/huggingface/metrics/f1/default/default_experiment-1-0.arrow.59it/s]
01/27/2022 09:30:48 - INFO - datasets.metric - Removing /home/ubuntu/.cache/huggingface/metrics/precision/default/default_experiment-1-0.arrow
01/27/2022 09:30:49 - INFO - datasets.metric - Removing /home/ubuntu/.cache/huggingface/metrics/recall/default/default_experiment-1-0.arrow
PRECISION:  {'precision': 0.7612903225806451}
RECALL:  {'recall': 0.7612903225806451}
F1:  {'f1': 0.7612903225806451}
{'eval_loss': 1.4658324718475342, 'eval_accuracy': 0.7612903118133545, 'eval_runtime': 30.0054, 'eval_samples_per_second': 46.492, 'eval_steps_per_second': 46.492, 'epoch': 3.0} 
**4th Epoch:**
1/27/2022 09:56:55 - INFO - datasets.metric - Removing /home/ubuntu/.cache/huggingface/metrics/f1/default/default_experiment-1-0.arrow.92it/s]
01/27/2022 09:56:56 - INFO - datasets.metric - Removing /home/ubuntu/.cache/huggingface/metrics/precision/default/default_experiment-1-0.arrow
01/27/2022 09:56:56 - INFO - datasets.metric - Removing /home/ubuntu/.cache/huggingface/metrics/recall/default/default_experiment-1-0.arrow
PRECISION:  {'precision': 0.7698924731182796}
RECALL:  {'recall': 0.7698924731182796}
F1:  {'f1': 0.7698924731182796}


## Environment info
!git clone https://github.com/huggingface/transformers
%cd transformers
!pip install .
!pip install -r /content/transformers/examples/pytorch/token-classification/requirements.txt
!pip install datasets

Comments:
Hi @Dhanachandra, 

We have tests for all our metrics and they work as expected: under the hood, we use scikit-learn implementations.

Maybe the cause is somewhere else. For example:
- Is it a binary or a multiclass or a multilabel classification? Default computation of these metrics is for binary classification; if you would like multiclass or multilabel, you should pass the corresponding parameters; see their documentation (e.g.: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_score.html) or code below:

https://huggingface.co/docs/datasets/using_metrics.html#computing-the-metric-scores

```python
In [1]: from datasets import load_metric

In [2]: precision = load_metric("precision")

In [3]: print(precision.inputs_description)

Args:
    predictions: Predicted labels, as returned by a model.
    references: Ground truth labels.
    labels: The set of labels to include when average != 'binary', and
        their order if average is None. Labels present in the data can
        be excluded, for example to calculate a multiclass average ignoring
        a majority negative class, while labels not present in the data will
        result in 0 components in a macro average. For multilabel targets,
        labels are column indices. By default, all labels in y_true and
        y_pred are used in sorted order.
    average: This parameter is required for multiclass/multilabel targets.
        If None, the scores for each class are returned. Otherwise, this
        determines the type of averaging performed on the data:
            binary: Only report results for the class specified by pos_label.
                This is applicable only if targets (y_{true,pred}) are binary.
            micro: Calculate metrics globally by counting the total true positives,
                false negatives and false positives.
            macro: Calculate metrics for each label, and find their unweighted mean.
                This does not take label imbalance into account.
            weighted: Calculate metrics for each label, and find their average
                weighted by support (the number of true instances for each label).
                This alters ‘macro’ to account for label imbalance; it can result
                in an F-score that is not between precision and recall.
            samples: Calculate metrics for each instance, and find their average
                (only meaningful for multilabel classification).
    sample_weight: Sample weights.

Returns:
    precision: Precision score.

Examples:

    >>> precision_metric = datasets.load_metric("precision")
    >>> results = precision_metric.compute(references=[0, 1], predictions=[0, 1])
    >>> print(results)
    {'precision': 1.0}

    >>> predictions = [0, 2, 1, 0, 0, 1]
    >>> references = [0, 1, 2, 0, 1, 2]
    >>> results = precision_metric.compute(predictions=predictions, references=references, average='macro')
    >>> print(results)
    {'precision': 0.2222222222222222}
    >>> results = precision_metric.compute(predictions=predictions, references=references, average='micro')
    >>> print(results)
    {'precision': 0.3333333333333333}
    >>> results = precision_metric.compute(predictions=predictions, references=references, average='weighted')
    >>> print(results)
    {'precision': 0.2222222222222222}
    >>> results = precision_metric.compute(predictions=predictions, references=references, average=None)
    >>> print(results)
    {'precision': array([0.66666667, 0.        , 0.        ])}
```