Generate: eta sampling numerical stability
Author: gante
Pull request: yes

# What does this PR do?

Minor numerical stability patch: before this change, the exception below was popping up sometimes, especially at lower numerical resolutions.

Compute entropy from logits instead -> no exception

<details>
  
  ```py
  │ /home/joao/transformers/src/transformers/generation/logits_process.py:423 in __call__            │
  │                                                                                                  │
  │   420 │   │   # Calculate the adaptive cutoff                                                    │
  │   421 │   │   probabilities = scores.softmax(dim=-1)                                             │
  │   422 │   │   print("probs > 0:", (probabilities > 0).sum(dim=1).max())                          │
  │ ❱ 423 │   │   entropy = torch.distributions.Categorical(probs=probabilities).entropy()           │
  │   424 │   │   eta = torch.min(self.epsilon, torch.sqrt(self.epsilon) * torch.exp(-entropy))[..   │
  │   425 │   │   indices_to_remove = probabilities < eta                                            │
  │   426                                                                                            │
  │                                                                                                  │
  │ /home/joao/hf/lib/python3.10/site-packages/torch/distributions/categorical.py:66 in __init__     │
  │                                                                                                  │
  │    63 │   │   self._param = self.probs if probs is not None else self.logits                     │
  │    64 │   │   self._num_events = self._param.size()[-1]                                          │
  │    65 │   │   batch_shape = self._param.size()[:-1] if self._param.ndimension() > 1 else torch   │
  │ ❱  66 │   │   super(Categorical, self).__init__(batch_shape, validate_args=validate_args)        │
  │    67 │                                                                                          │
  │    68 │   def expand(self, batch_shape, _instance=None):                                         │
  │    69 │   │   new = self._get_checked_instance(Categorical, _instance)                           │
  │                                                                                                  │
  │ /home/joao/hf/lib/python3.10/site-packages/torch/distributions/distribution.py:56 in __init__    │
  │                                                                                                  │
  │    53 │   │   │   │   value = getattr(self, param)                                               │
  │    54 │   │   │   │   valid = constraint.check(value)                                            │
  │    55 │   │   │   │   if not valid.all():                                                        │
  │ ❱  56 │   │   │   │   │   raise ValueError(                                                      │
  │    57 │   │   │   │   │   │   f"Expected parameter {param} "                                     │
  │    58 │   │   │   │   │   │   f"({type(value).__name__} of shape {tuple(value.shape)}) "         │
  │    59 │   │   │   │   │   │   f"of distribution {repr(self)} "                                   │
  ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
  ValueError: Expected parameter probs (Tensor of shape (16, 32128)) of distribution Categorical(probs: torch.Size([16, 32128])) to satisfy the
  constraint Simplex(), but found invalid values:
  tensor([[0.0000e+00, 3.9062e-03, 0.0000e+00,  ..., 0.0000e+00, 0.0000e+00,
           0.0000e+00],
          [0.0000e+00, 3.4485e-03, 0.0000e+00,  ..., 0.0000e+00, 0.0000e+00,
           0.0000e+00],
          [0.0000e+00, 7.6953e-01, 0.0000e+00,  ..., 0.0000e+00, 0.0000e+00,
           0.0000e+00],
          ...,
          [0.0000e+00, 6.3477e-03, 0.0000e+00,  ..., 0.0000e+00, 0.0000e+00,
           0.0000e+00],
          [0.0000e+00, 2.8381e-03, 0.0000e+00,  ..., 0.0000e+00, 0.0000e+00,
           0.0000e+00],
          [0.0000e+00, 6.3479e-06, 0.0000e+00,  ..., 0.0000e+00, 0.0000e+00,
           0.0000e+00]], device='cuda:0', dtype=torch.bfloat16)
  ```

</details>

Comments:
_The documentation is not available anymore as the PR was closed or merged._