Repository: huggingface/lerobot

Issue:
[HIL SERL] multiprocessing Queue.get_nowait is not getting all the items in the queue
### System Info

```Shell
- `lerobot` version: 0.1.0
- Platform: Linux-6.11.0-29-generic-x86_64-with-glibc2.39
- Python version: 3.10.13
- Huggingface_hub version: 0.30.2
- Dataset version: 3.5.1
- Numpy version: 2.2.6
- PyTorch version (GPU?): 2.6.0+cu124 (True)
- Cuda version: 12040
- Using GPU in script?: 
`python -m lerobot.scripts.rl.learner --config_path ...`
`python -m lerobot.scripts.rl.actor --config_path ...`
```

### Information

- [ ] One of the scripts in the examples/ folder of LeRobot
- [ ] My own task or dataset (give details below)

### Reproduction

Sometimes the `parameters_queue` on the actor keep growing, and can consume all the RAM.
_It can also potentially means that the actor is not getting the latest policy(did not reviewed this)._

Here can be seen that the "Policy(actor) parameters queue size" keeps growing. The queue size on the learner (Optimization prefix) looks stable.
<img width="2985" height="315" alt="Image" src="https://github.com/user-attachments/assets/94062e8c-6628-4eda-807e-d2ad416ecbb8" />

Similar issue was reported for cases when the queues are used with big items in https://bugs.python.org/issue23582.

I updated the code to drain the queue in `get_last_item_from_queue`, and the memory usage was reduced and become  more stable. The red line is without fix, other lines with the fix code bellow.

<img width="426" height="268" alt="Image" src="https://github.com/user-attachments/assets/100c63ec-554f-4d54-8d86-f17389b892bf" />

```
    # Drain queue and keep only the most recent parameters
    while queue.qsize():
        try:
            while True:
                item = queue.get_nowait()
        except Empty:
            pass
```


### Expected behavior

The parameters_queue should be drain complete to get the latest policy and to avoid OOM errors.
I will open a PR soon

cc @AdilZouitine 


Discussion:

Patch:
diff --git a/src/lerobot/utils/queue.py b/src/lerobot/utils/queue.py
index ceb30e2bff..864d798aca 100644
--- a/src/lerobot/utils/queue.py
+++ b/src/lerobot/utils/queue.py
@@ -14,6 +14,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import platform
+from contextlib import suppress
 from queue import Empty
 from typing import Any
 
@@ -30,10 +32,21 @@ def get_last_item_from_queue(queue: Queue, block=True, timeout: float = 0.1) ->
         item = None
 
     # Drain queue and keep only the most recent parameters
-    try:
-        while True:
+    if platform.system() == "Darwin":
+        # On Mac, avoid using `qsize` due to unreliable implementation.
+        # There is a comment on `qsize` code in the Python source:
+        # Raises NotImplementedError on Mac OSX because of broken sem_getvalue()
+        try:
+            while True:
+                item = queue.get_nowait()
+        except Empty:
+            pass
+
+        return item
+
+    # Details about using qsize in https://github.com/huggingface/lerobot/issues/1523
+    while queue.qsize() > 0:
+        with suppress(Empty):
             item = queue.get_nowait()
-    except Empty:
-        pass
 
     return item


Test patch:
diff --git a/tests/utils/test_queue.py b/tests/utils/test_queue.py
index 0a0d21770e..6e42acdb7b 100644
--- a/tests/utils/test_queue.py
+++ b/tests/utils/test_queue.py
@@ -18,6 +18,8 @@
 import time
 from queue import Queue
 
+from torch.multiprocessing import Queue as TorchMPQueue
+
 from lerobot.utils.queue import get_last_item_from_queue
 
 
@@ -46,6 +48,20 @@ def test_get_last_item_multiple_items():
     assert queue.empty()
 
 
+def test_get_last_item_multiple_items_with_torch_queue():
+    """Test getting the last item when queue has multiple items."""
+    queue = TorchMPQueue()
+    items = ["first", "second", "third", "fourth", "last"]
+
+    for item in items:
+        queue.put(item)
+
+    result = get_last_item_from_queue(queue)
+
+    assert result == "last"
+    assert queue.empty()
+
+
 def test_get_last_item_different_types():
     """Test with different data types in the queue."""
     queue = Queue()