Repository: httpie/cli (Python)

Issue:
Server-sent events explicitly requires --stream
If I have a [server-sent event](https://en.wikipedia.org/wiki/Server-sent_events) source and try to simply do

```
http get http://localhost:1729/stream/all
```

I get output that looks something like this:

```
HTTP/1.1 200 OK
Cache-Control: no-cache
Content-Type: text/event-stream
Date: Sat, 29 Aug 2015 12:18:00 GMT
Server: TornadoServer/4.2.1
Transfer-Encoding: chunked

```

followed by none of the streamed data. This can be worked around by setting the `--stream` flag. However, better behavior would be to automatically enable streaming support when `Content-Type` is `text/event-stream`.



Discussion:
Actually, the best behaviour is to auto-enable `--stream` when you see `Transfer-Encoding: chunked` because the length is indeterminate. That said, this is going to be near impossible to do for httpie. An example of what httpie is doing under the covers is

``` py
import requests
r = requests.get('http://localhost:1729/stream/all')
print_headers(r)
print_body(r)
```

What it does with `--stream` is:

``` py
import requests
r = requests.get('http://localhost:1729/stream/all', stream=True)
print_headers(r)
print_body(r)
```

This means that by the time httpie has received the headers it's too late to begin streaming. httpie would need to perform a `HEAD` request for every URL to determine whether or not to automatically enable `--stream` for the user which would be wildly inefficient. While this is certainly a desirable feature, I'm not sure it's practical. 

The one way to acommodate this would be for httpie to always pass `stream=True` to requests and if `--stream` isn't provided and the headers aren't something that should require streaming, then it reads the whole content into memory (accessing the `content` attribute will take care of that). Otherwise, it streams appropriately.

I'm not sure if @jkbrzt wants httpie to absorb some logic from it's dependencies though just to perform some magic.

If a proper fix is not feasible for the reasons @sigmavirus24 details above, perhaps issuing some kind of warning at the console could be an option. Presumably, `httpie` could notice after the fact that the response's `Content-Type:` is `text/event-stream` and then warn the user if they did not explicitly provide `--stream`.

I ran into this issue and thought I must be doing something wrong on the server side. For example, like @mivade, I was doing something like this:

```
$ http get http://localhost:8080/interval Accept:text/event-stream
HTTP/1.1 200 OK
Content-Type: text/event-stream
transfer-encoding: chunked


[hang forever]
```

Then—after considerable time debugging the server side—I tried the same request with `curl`, and saw that it just worked:

```
$ curl http://localhost:8080/interval -H Accept:text/event-stream
data:hello 0

data:hello 1

data:hello 2
```

Only then did I think to search through HTTPie's GitHub issues, at which point I found on this issue. Emitting a warning message as suggested above would save others from going through the same effort in the future. Also, if the user is explicitly specifying an `Accept:` header with a stream-based media type (as I do in my examples above), that might be a chance for HTTPie to auto-enable `--stream`.
> If a proper fix is not feasible for the reasons @sigmavirus24 details above, perhaps issuing some kind of warning at the console could be an option. Presumably, httpie could notice after the fact that the response's Content-Type: is text/event-stream and then warn the user if they did not explicitly provide --stream.

So, the underlying transport library, will try to read the entire response before giving control back to HTTPie. 

To solve this, the *best* course of action is for HTTPie to always stream responses unless told not to. This will not incur a performance penalty for HTTPie.
Hi, I'm a newbie to open source projects. I would like to contribute to make HTTPie to automatically recognize server sent events. Could you please provide me with more info and can I work on this issue?
Any news on this? I've the same problem... a simple warning would saved hours of debugging.
To people who want to work on this, it looks like httpie [already streams responses by default](https://github.com/jakubroztocil/httpie/blob/87e661c5f165e710ecd0863c6e3529d0a131da0d/httpie/client.py#L157). The trick is now to find where to add support for auto-enabling `--stream` when the content type matches one of the content-types that should be streamed.
> the best behaviour is to auto-enable --stream when you see Transfer-Encoding: chunked because the length is indeterminate

@sigmavirus24 great suggestion, I'll look into that.
Hey all, 
Came across this issue when I was looking into why httpie does not show the SSE. `--stream` indeed solved it.
I did some tinkering in httpie code and noticed a potential location it can be handled from the response.
I would like to work on it if it is still relevant and maintainers of httpie are onboard. :)

cc @jakubroztocil 



Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 7cdcbb195e..5f040ab15d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -13,8 +13,10 @@ This project adheres to [Semantic Versioning](https://semver.org/).
 - Added support for _sending_ multiple HTTP header lines with the same name. ([#130](https://github.com/httpie/httpie/issues/130))
 - Added support for _receiving_ multiple HTTP headers lines with the same name. ([#1207](https://github.com/httpie/httpie/issues/1207))
 - Added support for basic JSON types on `--form`/`--multipart` when using JSON only operators (`:=`/`:=@`). ([#1212](https://github.com/httpie/httpie/issues/1212))
+- Added support for automatically enabling `--stream` when `Content-Type` is `text/event-stream`. ([#376](https://github.com/httpie/httpie/issues/376))
 - Broken plugins will no longer crash the whole application. ([#1204](https://github.com/httpie/httpie/issues/1204))
 
+
 ## [2.6.0](https://github.com/httpie/httpie/compare/2.5.0...2.6.0) (2021-10-14)
 
 [What’s new in HTTPie 2.6.0 →](https://httpie.io/blog/httpie-2.6.0)
diff --git a/httpie/output/writer.py b/httpie/output/writer.py
index 4650264db6..cd3eec979b 100644
--- a/httpie/output/writer.py
+++ b/httpie/output/writer.py
@@ -2,6 +2,7 @@
 import errno
 from typing import IO, TextIO, Tuple, Type, Union
 
+from ..cli.dicts import HTTPHeadersDict
 from ..context import Environment
 from ..models import (
     HTTPRequest,
@@ -110,6 +111,7 @@ def build_output_stream_for_message(
         env=env,
         args=args,
         message_type=message_type,
+        headers=requests_message.headers
     )
     yield from stream_class(
         msg=message_type(requests_message),
@@ -128,16 +130,23 @@ def get_stream_type_and_kwargs(
     env: Environment,
     args: argparse.Namespace,
     message_type: Type[HTTPMessage],
+    headers: HTTPHeadersDict,
 ) -> Tuple[Type['BaseStream'], dict]:
     """Pick the right stream type and kwargs for it based on `env` and `args`.
 
     """
+    is_stream = args.stream
+    if not is_stream and message_type is HTTPResponse:
+        # If this is a response, then check the headers for determining
+        # auto-streaming.
+        is_stream = headers.get('Content-Type') == 'text/event-stream'
+
     if not env.stdout_isatty and not args.prettify:
         stream_class = RawStream
         stream_kwargs = {
             'chunk_size': (
                 RawStream.CHUNK_SIZE_BY_LINE
-                if args.stream
+                if is_stream
                 else RawStream.CHUNK_SIZE
             )
         }
@@ -152,7 +161,7 @@ def get_stream_type_and_kwargs(
                 'encoding_overwrite': args.response_charset,
             })
         if args.prettify:
-            stream_class = PrettyStream if args.stream else BufferedPrettyStream
+            stream_class = PrettyStream if is_stream else BufferedPrettyStream
             stream_kwargs.update({
                 'conversion': Conversion(),
                 'formatting': Formatting(


Test patch:
diff --git a/tests/test_stream.py b/tests/test_stream.py
index 002b245af0..55e000a6dd 100644
--- a/tests/test_stream.py
+++ b/tests/test_stream.py
@@ -2,6 +2,7 @@
 
 import pytest
 import responses
+from unittest.mock import Mock
 
 from httpie.compat import is_windows
 from httpie.cli.constants import PRETTY_MAP
@@ -107,3 +108,28 @@ def test_redirected_stream(httpbin):
     r = http('--pretty=none', '--stream', '--verbose', 'GET',
              httpbin.url + '/get', env=env)
     assert BIN_FILE_CONTENT in r
+
+
+# /drip endpoint produces 3 individual lines,
+# if we set text/event-stream HTTPie should stream
+# it by default. Otherwise, it will buffer and then
+# print.
+@pytest.mark.parametrize('extras, expected', [
+    (
+        ['Accept:text/event-stream'],
+        3
+    ),
+    (
+        ['Accept:text/plain'],
+        1
+    )
+])
+def test_auto_streaming(http_server, extras, expected):
+    env = MockEnvironment()
+    env.stdout.write = Mock()
+    http(http_server + '/drip', *extras, env=env)
+    assert len([
+        call_arg
+        for call_arg in env.stdout.write.call_args_list
+        if b'test' in call_arg[0][0]
+    ]) == expected
diff --git a/tests/utils/http_server.py b/tests/utils/http_server.py
index 42be768b46..f09e06c2c2 100644
--- a/tests/utils/http_server.py
+++ b/tests/utils/http_server.py
@@ -36,6 +36,22 @@ def get_headers(handler):
     handler.end_headers()
 
 
+@TestHandler.handler('GET', '/drip')
+def chunked_drip(handler):
+    handler.send_response(200)
+    accept = handler.headers.get('Accept')
+    if accept is not None:
+        handler.send_header('Content-Type', accept)
+    handler.send_header('Transfer-Encoding', 'chunked')
+    handler.end_headers()
+
+    for _ in range(3):
+        body = 'test\n'
+        handler.wfile.write(f'{len(body):X}\r\n{body}\r\n'.encode('utf-8'))
+
+    handler.wfile.write('0\r\n\r\n'.encode('utf-8'))
+
+
 @pytest.fixture(scope="function")
 def http_server():
     """A custom HTTP server implementation for our tests, that is