Repository: huggingface/trl

Issue:
Can not import name top_k_top_p_filtering
Hello everyone, I just came across this kind of error, I cannot import trl because of this kind of error: 
```ImportError: cannot import name 'top_k_top_p_filtering' from 'transformers' (/usr/local/lib/python3.10/dist-packages/transformers/__init__.py)```
although I upgrade transformers library, it remained not working

https://github.com/huggingface/trl/blob/22b4f548f4954319ece6f17cab226a26e2db65be/trl/core.py#L25


Discussion:
Same here I encountered the same error this week. Last week can imported SFTTrainer without this error. Another developer also bumped into this error this week see this closed [issue](https://github.com/huggingface/trl/issues/6)
![git](https://github.com/huggingface/trl/assets/8589224/97c935af-b310-4cb5-a614-71d3270e07f4)


Make sure that you have transformers version 4.38.2. Then, importing 'top_k_top_p_filtering' from"transformers.generation.utils" should work.
I would like to continue finetuning starcoder2. So I have to install transformers from the source with 
!pip install git+https://github.com/huggingface/transformers.git
I already SFT the model, but now I cannot import DPOTrainer. Using transformers 4.38.2 is not feasible as it doesn't recognize starcoder2. It does however solve the issue with DPOTrainer import -- but I need the newest version, which I think is 4.39.*

I tried to install top_k_top_p_filtering (cannot install DPOTRainer becuse of this) as suggested above but I get an error

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
[<ipython-input-8-22d4c10675d7>](https://localhost:8080/#) in <cell line: 1>()
----> 1 from transformers.generation.utils import top_k_top_p_filtering

ImportError: cannot import name 'top_k_top_p_filtering' from 'transformers.generation.utils' (/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py)

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.

What is the workaround for installing DPOTrainer for the latest transformer version? Thanks
 

Resolving patch:
diff --git a/trl/core.py b/trl/core.py
--- a/trl/core.py
+++ b/trl/core.py
@@ -22,7 +22,7 @@
 import torch.nn as nn
 import torch.nn.functional as F
 from torch.nn.utils.rnn import pad_sequence
-from transformers import top_k_top_p_filtering
+from transformers.generation import TopKLogitsWarper, TopPLogitsWarper
 
 from .import_utils import is_npu_available, is_xpu_available
 
@@ -36,6 +36,42 @@
 WANDB_PADDING = -1
 
 
+def top_k_top_p_filtering(
+    logits: torch.FloatTensor,
+    top_k: int = 0,
+    top_p: float = 1.0,
+    filter_value: float = -float("Inf"),
+    min_tokens_to_keep: int = 1,
+) -> torch.FloatTensor:
+    """
+    Filter a distribution of logits using top-k and/or nucleus (top-p) filtering.
+
+    Args:
+        logits: logits distribution shape (batch size, vocabulary size)
+        top_k (`int`, *optional*, defaults to 0):
+            If > 0, only keep the top k tokens with highest probability (top-k filtering)
+        top_p (`float`, *optional*, defaults to 1.0):
+            If < 1.0, only keep the top tokens with cumulative probability >= top_p (nucleus filtering). Nucleus
+            filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751)
+        min_tokens_to_keep (`int`, *optional*, defaults to 1):
+            Minimumber of tokens we keep per batch example in the output.
+
+    From: https://gist.github.com/thomwolf/1a5a29f6962089e871b94cbd09daf317
+    """
+
+    if top_k > 0:
+        logits = TopKLogitsWarper(top_k=top_k, filter_value=filter_value, min_tokens_to_keep=min_tokens_to_keep)(
+            None, logits
+        )
+
+    if 0 <= top_p <= 1.0:
+        logits = TopPLogitsWarper(top_p=top_p, filter_value=filter_value, min_tokens_to_keep=min_tokens_to_keep)(
+            None, logits
+        )
+
+    return logits
+
+
 def flatten_dict(nested: Dict, sep: str = "/") -> Dict:
     """Flatten dictionary and concatenate nested keys with separator."""
 


Test patch: