Classification metrics docs incorrectly state they work with logits
Currently, all classification metrics (ie. https://pytorch-lightning.readthedocs.io/en/stable/metrics.html#accuracy) state:
Accepts logits from a model output or integer class values in prediction..
In the parameter section, it states:
threshold (float) – Threshold value for binary or multi-label logits. default: 0.5
This suggests that predictions have already passed through a sigmoid at this stage, which logits have not.
The default threshold of 0.5 does not make sense with logits, or more correctly, raw model predictions (before sigmoid/softmax).
In #5340, the code around Accuracy was changed to say preds instead of logits, but the documentation remains to state logits.
I see two options (explained in the case of binary classification):

Continue accepting logits, but making explicit that inside the metric either the predictions will be passed through a sigmoid, or the threshold will be passed through a logit function [it is important that the documentation makes explicit that the given threshold still needs to be between 0 and 1 then].
The documentation changes from logits to probabilities, and explicitly states that one needs to apply sigmoid/softmax before passing predictions into the metric (this will be the case when users use CrossEntropyLoss or BCEWithLogitsLoss)

Personally, I would prefer a solution that accepts a parameter that can work with raw model outputs, as that would work well together with the more efficient BCEWithLogitsLoss/CrossEntropyLoss. This could also be a flag, like is_logit=False, which would apply sigmoid/softmax when True. That would also be required if one wants to be able to interpret thresholds returned by PrecisionRecallCurve.