Repository: Gallopsled/pwntools

Issue:
`tube.readline()` skips the last line if it's not terminated with a newline
I recently missed a flag in a CTF, because it seems that `tube.readline()` returns data only if it actually ended with a newline. If an EOF happens (even after some data was already read) it throws `EOFError` instead of returning the line.
I'm not sure how the API should ideally look like, but it seems quite dangerous in the current form, because e.g. this simple loop:
```Python
try:
    print(r.readline(keepends=False))
except EOFError:
    pass
```
will silently hide the last line of data if it's not newline-terminated, despite the code looking correct. Also, this quirk doesn't seem to be documented currently.

Additional argument why it's confusing: the standard recv/send semantics, probably in all runtimes/languages, raises EOF exception / signal / 0 return only if all the data was already consumed and someone retried reading.



Discussion:
Hm, the docs define a "line" as a sequence of bytes terminated by the `tube.newline` sequence.

Is it really a line if it doesn't end with the desired newline character? I guess eof is a special case  but having `recvline` not receive a line sounds weird. Maybe return everything buffered on eof but that feels like a breaking change?

Your usecase sounds like a job for `tube.stream()`.
If you think it's relevant, GLIBC's readline() considers EOF a newline if data preceded it 🤷‍♂️
True, it's better to not lose data. @Arusekk ?

Resolving patch:
diff --git a/pwnlib/context/__init__.py b/pwnlib/context/__init__.py
--- a/pwnlib/context/__init__.py
+++ b/pwnlib/context/__init__.py
@@ -367,6 +367,7 @@ class ContextType(object):
         'randomize': False,
         'rename_corefiles': True,
         'newline': b'\n',
+        'throw_eof_on_incomplete_line': None,
         'noptrace': False,
         'os': 'linux',
         'proxy': None,
@@ -1490,6 +1491,25 @@ def newline(self, v):
         # circular imports
         from pwnlib.util.packing import _need_bytes
         return _need_bytes(v)
+    
+    @_validator
+    def throw_eof_on_incomplete_line(self, v):
+        """Whether to raise an :class:`EOFError` if an EOF is received before a newline in ``tube.recvline``.
+
+        Controls if an :class:`EOFError` is treated as newline in ``tube.recvline`` and similar functions
+        and whether a warning should be logged about it.
+
+        Possible values are:
+
+        - ``True``: Raise an :class:`EOFError` if an EOF is received before a newline.
+        - ``False``: Return the data received so far if an EOF is received
+          before a newline without logging a warning.
+        - ``None``: Return the data received so far if an EOF is received
+          before a newline and log a warning.
+
+        Default value is ``None``.
+        """
+        return v if v is None else bool(v)
 
 
     @_validator
diff --git a/pwnlib/tubes/tube.py b/pwnlib/tubes/tube.py
--- a/pwnlib/tubes/tube.py
+++ b/pwnlib/tubes/tube.py
@@ -467,19 +467,31 @@ def recvline(self, keepends=True, timeout=default):
         Receive a single line from the tube.
 
         A "line" is any sequence of bytes terminated by the byte sequence
-        set in :attr:`newline`, which defaults to ``'\n'``.
+        set in :attr:`newline`, which defaults to ``b'\n'``.
+
+        If the connection is closed (:class:`EOFError`) before a newline
+        is received, the buffered data is returned by default and a warning
+        is logged. If the buffer is empty, an :class:`EOFError` is raised.
+        This behavior can be changed by setting :meth:`pwnlib.context.ContextType.throw_eof_on_incomplete_line`.
 
         If the request is not satisfied before ``timeout`` seconds pass,
-        all data is buffered and an empty string (``''``) is returned.
+        all data is buffered and an empty byte string (``b''``) is returned.
 
         Arguments:
             keepends(bool): Keep the line ending (:const:`True`).
             timeout(int): Timeout
 
+        Raises:
+            :class:`EOFError`: The connection closed before the request
+                                 could be satisfied and the buffer is empty
+
         Return:
             All bytes received over the tube until the first
             newline ``'\n'`` is received.  Optionally retains
-            the ending.
+            the ending. If the connection is closed before a newline
+            is received, the remaining data received up to this point
+            is returned.
+
 
         Examples:
 
@@ -494,8 +506,31 @@ def recvline(self, keepends=True, timeout=default):
             >>> t.newline = b'\r\n'
             >>> t.recvline(keepends = False)
             b'Foo\nBar'
+            >>> t = tube()
+            >>> def _recv_eof(n):
+            ...     if not _recv_eof.throw:
+            ...         _recv_eof.throw = True
+            ...         return b'real line\ntrailing data'
+            ...     raise EOFError
+            >>> _recv_eof.throw = False
+            >>> t.recv_raw = _recv_eof
+            >>> t.recvline()
+            b'real line\n'
+            >>> t.recvline()
+            b'trailing data'
+            >>> t.recvline() # doctest: +ELLIPSIS
+            Traceback (most recent call last):
+            ...
+            EOFError
         """
-        return self.recvuntil(self.newline, drop = not keepends, timeout = timeout)
+        try:
+            return self.recvuntil(self.newline, drop = not keepends, timeout = timeout)
+        except EOFError:
+            if not context.throw_eof_on_incomplete_line and self.buffer.size > 0:
+                if context.throw_eof_on_incomplete_line is None:
+                    self.warn_once('EOFError during recvline. Returning buffered data without trailing newline.')
+                return self.buffer.get()
+            raise
 
     def recvline_pred(self, pred, keepends=False, timeout=default):
         r"""recvline_pred(pred, keepends=False) -> bytes


Test patch: