Repository: aio-libs/aiohttp

Issue:
BaseRequest.http_range returns the wrong slice when "start" is missing
## Long story short

<!-- Please describe your problem and why the fix is important. -->
When streaming range based requests from file, the method [http_range](
https://docs.aiohttp.org/en/stable/web_reference.html#aiohttp.web.BaseRequest.http_range) comes quite handy, unfortunately negative indexes don't work as expected.

The offending line 442 is in [web_request.py](https://github.com/aio-libs/aiohttp/blame/20928e1d6ba4887b9137eacd8ad7dda2dbbb09e6/aiohttp/web_request.py#L442) it should be "start = -end" instead of "end = -end"

Other valid ranges are described in https://tools.ietf.org/html/rfc7233#section-2.1 
I don't know if it's allowed to just return a 416 on the more exotic ones though:)


## Expected behaviour
[Request Header]
Range: bytes=-5678

[BaseRequest.http_range]
slice(start=-5678, stop=None, step=1)

<!-- What is the behaviour you expect? -->

## Actual behaviour
[Request Header]
Range: bytes=-5678

[BaseRequest.http_range]
slice(start=None, stop=-5678, step=1)
<!-- What's actually happening? -->

## Steps to reproduce

<!-- Please describe steps to reproduce the issue.
     If you have a script that does that please include it here within
     markdown code markup -->
run an aiohttp server and request a resource using the range header.
(do I have to post the whole setup, code included?)

## Your environment

<!-- Describe the environment you have that lead to your issue.
     This includes aiohttp version, OS, proxy server and other bits that
     are related to your case. 
     
     IMPORTANT: aiohttp is both server framework and client library.
     For getting rid of confusing please put 'server', 'client' or 'both'
     word here.
     -->
Running aiohttp 1.2.0 on debian 9 (stretch) ... I know, I know :)


Discussion:
Would you make a pull request?
While what @Fahrradkette reported would be a nice and clean pull request to fix (and if he doesn't have the time/need I can take a stab at that) the problem is deeper than what is described in this ticket.

According to the RFC https://tools.ietf.org/html/rfc7233 section 2.1 the following are all valid range requests (when concerning bytes):

bytes=0-499
bytes=500-999
bytes=-500
bytes=9500-

The above can all be easily supported and are as you expect. However the following are also valid:

bytes=500-600,601-999
bytes=500-700,601-999

Where the response would be a multipart with those byte ranges. However currently in the above mentioned function only a single slice can be returned :)

What are your ideas on supporting the entire Range grammar which would likely be much more complex versus just fixing the first part of it. 

My opinion is to either support all of Range or none.
I doubt if multi-range requests are wide spread. Lets keep the single range for simplicity.
Multiranges require a new property anyway, we cannot do it with existing property without breaking backward compatibility.
Raising an exception in case of multirange is pretty fibe.
In that case if @Fahrradkette wants to create a pull request they can, otherwise I can.

Resolving patch:
diff --git a/aiohttp/web_fileresponse.py b/aiohttp/web_fileresponse.py
--- a/aiohttp/web_fileresponse.py
+++ b/aiohttp/web_fileresponse.py
@@ -193,9 +193,9 @@ async def prepare(self, request):
         # If a range request has been made, convert start, end slice notation
         # into file pointer offset and count
         if start is not None or end is not None:
-            if start is None and end < 0:  # return tail of file
-                start = file_size + end
-                count = -end
+            if start < 0 and end is None:  # return tail of file
+                start = file_size + start
+                count = file_size - start
             else:
                 count = (end or file_size) - start
 
diff --git a/aiohttp/web_request.py b/aiohttp/web_request.py
--- a/aiohttp/web_request.py
+++ b/aiohttp/web_request.py
@@ -439,7 +439,8 @@ def http_range(self, *, _RANGE=hdrs.RANGE):
 
             if start is None and end is not None:
                 # end with no start is to return tail of content
-                end = -end
+                start = -end
+                end = None
 
             if start is not None and end is not None:
                 # end is inclusive in range header, exclusive for slice
@@ -450,6 +451,7 @@ def http_range(self, *, _RANGE=hdrs.RANGE):
 
             if start is end is None:  # No valid range supplied
                 raise ValueError('No start or end of range specified')
+
         return slice(start, end, 1)
 
     @property


Test patch:
diff --git a/tests/test_web_request.py b/tests/test_web_request.py
--- a/tests/test_web_request.py
+++ b/tests/test_web_request.py
@@ -110,6 +110,58 @@ def test_content_length():
     assert 123 == req.content_length
 
 
+def test_range_to_slice_head():
+    def bytes_gen(size):
+        for i in range(size):
+            yield i % 256
+    payload = bytearray(bytes_gen(10000))
+    req = make_mocked_request(
+        'GET', '/',
+        headers=CIMultiDict([('RANGE', 'bytes=0-499')]),
+        payload=payload)
+    assert isinstance(req.http_range, slice)
+    assert req.content[req.http_range] == payload[:500]
+
+
+def test_range_to_slice_mid():
+    def bytes_gen(size):
+        for i in range(size):
+            yield i % 256
+    payload = bytearray(bytes_gen(10000))
+    req = make_mocked_request(
+        'GET', '/',
+        headers=CIMultiDict([('RANGE', 'bytes=500-999')]),
+        payload=payload)
+    assert isinstance(req.http_range, slice)
+    assert req.content[req.http_range] == payload[500:1000]
+
+
+def test_range_to_slice_tail_start():
+    def bytes_gen(size):
+        for i in range(size):
+            yield i % 256
+    payload = bytearray(bytes_gen(10000))
+    req = make_mocked_request(
+        'GET', '/',
+        headers=CIMultiDict([('RANGE', 'bytes=9500-')]),
+        payload=payload)
+    assert isinstance(req.http_range, slice)
+    assert req.content[req.http_range] == payload[-500:]
+
+
+def test_range_to_slice_tail_stop():
+    def bytes_gen(size):
+        for i in range(size):
+            yield i % 256
+    payload = bytearray(bytes_gen(10000))
+    req = make_mocked_request(
+        'GET', '/',
+        headers=CIMultiDict([('RANGE', 'bytes=-500')]),
+        payload=payload)
+    assert isinstance(req.http_range, slice)
+    assert req.content[req.http_range] == payload[-500:]
+
+
 def test_non_keepalive_on_http10():
     req = make_mocked_request('GET', '/', version=HttpVersion(1, 0))
     assert not req.keep_alive