Repository: gabrieldemarmiesse/python-on-whales (language: python)

Issue:
Missing lines from logs output
When using docker.container.logs, some lines are missing in the output

For example with postgres:

```
$ docker run -e POSTGRES_PASSWORD=password --detach postgres
8fb3fd4da45f0294ed183a988b27bc728407e10955dfa04fdea9869083e16304

$ docker logs --tail 8 8fb3fd4da45f0294ed183a988b27bc728407e10955dfa04fdea9869083e16304
PostgreSQL init process complete; ready for start up.

2021-07-18 16:03:21.262 UTC [1] LOG:  starting PostgreSQL 13.3 (Debian 13.3-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
2021-07-18 16:03:21.262 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2021-07-18 16:03:21.262 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2021-07-18 16:03:21.268 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-07-18 16:03:21.274 UTC [65] LOG:  database system was shut down at 2021-07-18 16:03:21 UTC
2021-07-18 16:03:21.281 UTC [1] LOG:  database system is ready to accept connections

```

The same command on python on whales only prints 1 line, all the LOGs after the ready for start up are missing:

```
>>> print(docker.container.logs("8fb3fd4da45f0294ed183a988b27bc728407e10955dfa04fdea9869083e16304", tail=8))
PostgreSQL init process complete; ready for start up.
```


Maybe that the missing lines are from stderr and a `return_stderr` is needed? (it is defaulted False in run and is not specified by the logs command)


Discussion / hints:
You are totally right. I forgot that the stderr of the container is sent to the stderr of the docker process. I'll take care of it. Hopefully it will be quick to fix.
Thank you for sending me so many bug reports, those bugs are not easy to find :) 
Thanks to you for fixing them!
I'm introducing python on whales onto my stacks to replace compose (the python version, used directly from a python wrapper) with swarm. It is about ten projects with several deployments managed by different maintainers... not a huge use case, but allows me to test the package in different real life conditions, so I'm glad to report feedback and bugs to improve this ultra useful package
We also replaced compose by swarm in my company. With swarmpit on top of it, it works like a charm. We use also python-on-whales for deployments and to automate maintenance (cleanups and scaling services). 

Fix patch:
diff --git a/python_on_whales/components/container/cli_wrapper.py b/python_on_whales/components/container/cli_wrapper.py
index 87744db7..99b419eb 100644
--- a/python_on_whales/components/container/cli_wrapper.py
+++ b/python_on_whales/components/container/cli_wrapper.py
@@ -867,7 +867,9 @@ def logs(
         tail: Optional[int] = None,
         timestamps: bool = False,
         until: Union[None, datetime, timedelta] = None,
-    ) -> str:
+        follow: bool = False,
+        stream: bool = False,
+    ) -> Union[str, Iterable[Tuple[str, bytes]]]:
         """Returns the logs of a container as a string.
 
         Alias: `docker.logs(...)`
@@ -883,20 +885,53 @@ def logs(
             timestamps: Put timestamps next to lines.
             until: Use a datetime or a timedelta to specify the upper date
                 limit for the logs.
+            follow: If `False` (the default), the logs returned are the logs up to the time
+                of the function call. If `True`, the logs of the container up to the time the
+                container stopped are displayed. Which means that if the container isn't stopped
+                yet, the function will continue until the container is stopped.
+                Which is why it is advised to use the `stream` option if you use the `follow` option.
+                Without `stream`, only a `str` will be returned, possibly much later in the
+                future. With `stream`, you'll be able to read the logs in real time.
+            stream: Similar to the `stream` argument of `docker.run`.
+                This function will then returns and iterator that will yield a
+                tuple `(source, content)` with `source` being `"stderr"` or
+                `"stdout"`. `content` is the content of the line as bytes.
+                Take a look at [the user guide](https://gabrieldemarmiesse.github.io/python-on-whales/user_guide/docker_run/#stream-the-output)
+                to have an example of the output.
 
         # Returns
-            `str`
+            `str` if `stream=False` (the default), `Iterable[Tuple[str, bytes]]`
+            if `stream=True`.
 
         # Raises
             `python_on_whales.exceptions.NoSuchContainer` if the container does not exists.
+
+        If you are a bit confused about `follow` and `stream`, here are some use cases.
+
+        * If you want to have the logs up to this point as a `str`, don't use those args.
+        * If you want to stream the output in real time, use `follow=True, stream=True`
+        * If you want the logs up to this point but you don't want to fit all the logs
+        in memory because they are too big, use `stream=True`.
         """
+
+        # first we verify that the container exists and raise an exception if not.
+        self.inspect(container)
+
         full_cmd = self.docker_cmd + ["container", "logs"]
         full_cmd.add_flag("--details", details)
         full_cmd.add_simple_arg("--since", format_time_arg(since))
         full_cmd.add_simple_arg("--tail", tail)
         full_cmd.add_flag("--timestamps", timestamps)
         full_cmd.add_simple_arg("--until", format_time_arg(until))
-        return run(full_cmd + [container])
+        full_cmd.add_flag("--follow", follow)
+        full_cmd.append(container)
+
+        iterator = stream_stdout_and_stderr(full_cmd)
+
+        if stream:
+            return iterator
+        else:
+            return "".join(x[1].decode() for x in iterator)
 
     def list(self, all: bool = False, filters: Dict[str, str] = {}) -> List[Container]:
         """List the containers on the host.


Test patch:
diff --git a/tests/python_on_whales/components/test_container.py b/tests/python_on_whales/components/test_container.py
index 5e4fce48..3ee98433 100644
--- a/tests/python_on_whales/components/test_container.py
+++ b/tests/python_on_whales/components/test_container.py
@@ -155,10 +155,67 @@ def test_container_remove():
 
 
 def test_simple_logs():
-    container = docker.run("busybox:1", ["echo", "dodo"], detach=True)
-    time.sleep(0.3)
-    output = docker.container.logs(container)
-    assert output == "dodo"
+    with docker.run("busybox:1", ["echo", "dodo"], detach=True) as c:
+        time.sleep(0.3)
+        output = docker.container.logs(c)
+        assert output == "dodo\n"
+
+
+def test_simple_logs_stderr():
+    with docker.run("busybox:1", ["sh", "-c", ">&2 echo dodo"], detach=True) as c:
+        time.sleep(0.3)
+        output = docker.container.logs(c)
+        assert output == "dodo\n"
+
+
+def test_simple_logs_do_not_follow():
+    with docker.run(
+        "busybox:1", ["sh", "-c", "sleep 3 && echo dodo"], detach=True
+    ) as c:
+        output = docker.container.logs(c, follow=False)
+        assert output == ""
+
+
+def test_simple_logs_follow():
+    with docker.run(
+        "busybox:1", ["sh", "-c", "sleep 3 && echo dodo"], detach=True
+    ) as c:
+        output = docker.container.logs(c, follow=True)
+        assert output == "dodo\n"
+
+
+def test_simple_logs_stream():
+    with docker.run("busybox:1", ["echo", "dodo"], detach=True) as c:
+        time.sleep(0.3)
+        output = list(docker.container.logs(c, stream=True))
+        assert output == [("stdout", b"dodo\n")]
+
+
+def test_simple_logs_stream_stderr():
+    with docker.run("busybox:1", ["sh", "-c", ">&2 echo dodo"], detach=True) as c:
+        time.sleep(0.3)
+        output = list(docker.container.logs(c, stream=True))
+        assert output == [("stderr", b"dodo\n")]
+
+
+def test_simple_logs_stream_stdout_and_stderr():
+    with docker.run(
+        "busybox:1", ["sh", "-c", ">&2 echo dodo && echo dudu"], detach=True
+    ) as c:
+        time.sleep(0.3)
+        output = list(docker.container.logs(c, stream=True))
+
+        # the order of stderr and stdout is not guaranteed.
+        assert set(output) == {("stderr", b"dodo\n"), ("stdout", b"dudu\n")}
+
+
+def test_simple_logs_stream_wait():
+    with docker.run(
+        "busybox:1", ["sh", "-c", "sleep 3 && echo dodo"], detach=True
+    ) as c:
+        time.sleep(0.3)
+        output = list(docker.container.logs(c, follow=True, stream=True))
+        assert output == [("stdout", b"dodo\n")]
 
 
 def test_rename():