Repository: huge-success/sanic
Pull request #1476 (2019-01-18 16:48:42)

add Request.not_grouped_args, deprecation warning Request.raw_args
`Request.raw_args` has a huge bug - it takes only the first value of the parameter.
In case of the URLs with `?key1=value1&key2=value2&key1=value3` querystring, `key1` will get `["value1", "value3"]` in `Request.args` and only `value1` in `Request.raw_args`. 

And you can't call it even raw - it is parsed and then reduced.

As it is no opportunity to return multiple equal keys using `dict` for the backward compatibility I created another property `Request.not_grouped_args` that returns list of tuples from `urllib.parse.parse_qsl` method.
I think it is better avoid using `raw_args` and cut it from the future versions.

codecov[bot]: # [Codecov](https://codecov.io/gh/huge-success/sanic/pull/1476?src=pr&el=h1) Report
> Merging [#1476](https://codecov.io/gh/huge-success/sanic/pull/1476?src=pr&el=desc) into [master](https://codecov.io/gh/huge-success/sanic/commit/99f34c9f5047dd6721e2d09eb3efbd982e844d2c?src=pr&el=desc) will **decrease** coverage by `0.17%`.
> The diff coverage is `76.92%`.

[![Impacted file tree graph](https://codecov.io/gh/huge-success/sanic/pull/1476/graphs/tree.svg?width=650&token=NqLbJHUqoR&height=150&src=pr)](https://codecov.io/gh/huge-success/sanic/pull/1476?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #1476      +/-   ##
==========================================
- Coverage   91.45%   91.28%   -0.18%     
==========================================
  Files          17       17              
  Lines        1732     1744      +12     
  Branches      329      332       +3     
==========================================
+ Hits         1584     1592       +8     
- Misses        123      124       +1     
- Partials       25       28       +3
```


| [Impacted Files](https://codecov.io/gh/huge-success/sanic/pull/1476?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sanic/request.py](https://codecov.io/gh/huge-success/sanic/pull/1476/diff?src=pr&el=tree#diff-c2FuaWMvcmVxdWVzdC5weQ==) | `98.18% <76.92%> (-1.34%)` | :arrow_down: |
| [sanic/router.py](https://codecov.io/gh/huge-success/sanic/pull/1476/diff?src=pr&el=tree#diff-c2FuaWMvcm91dGVyLnB5) | `95.43% <0%> (-0.46%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huge-success/sanic/pull/1476?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huge-success/sanic/pull/1476?src=pr&el=footer). Last update [99f34c9...46c181e](https://codecov.io/gh/huge-success/sanic/pull/1476?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

Amanit: If I understand well the coverage test is not passed because of the
```
if self.app.debug:
    warnings.simplefilter("default")
```
Should I really test it?
ahopkins: I will look over the PR more closet later. As for the warning question, that looks fine to add a noqa
Amanit: @ahopkins noqa did not affected the coverage, I have set # pragma: no cover.
But it is still - 0.01% of coverage :)
yunstanford: thanks for the PR! i'm thinking about the name, it's not very intuitive right now..
Amanit: @ahopkins for now `raw_args` is not working properly, but there is no possibility to fix it without changing the output type. I think that for now it is just enough to not mention it in the documentation and show a Warning on use. 
Amanit: @ahopkins @yunstanford Any suggestions on this PR?
Amanit: > Are we going to deprecate raw_args? Let's address this and make a decision. If we are, the next thing we need to talk about is how to communicate that.

@ahopkins who we need to call to make this decision? :)
sjsadowski: @ahopkins @harshanarayana @huge-success/sanic-core-devs are we comfortable accepting this PR at this point?

Patch:
diff --git a/docs/sanic/api_reference.rst b/docs/sanic/api_reference.rst
index 5ca7556a..8b8dc04a 100644
--- a/docs/sanic/api_reference.rst
+++ b/docs/sanic/api_reference.rst
@@ -20,6 +20,15 @@ sanic.blueprints module
     :undoc-members:
     :show-inheritance:
 
+sanic.blueprint_group module
+----------------------------
+
+.. automodule:: sanic.blueprint_group
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+
 sanic.config module
 -------------------
 
diff --git a/docs/sanic/blueprints.md b/docs/sanic/blueprints.md
index e38cf4b1..6edbf1a8 100644
--- a/docs/sanic/blueprints.md
+++ b/docs/sanic/blueprints.md
@@ -127,7 +127,7 @@ Blueprints have almost the same functionality as an application instance.
 WebSocket handlers can be registered on a blueprint using the `@bp.websocket`
 decorator or `bp.add_websocket_route` method.
 
-### Middleware
+### Blueprint Middleware
 
 Using blueprints allows you to also register middleware globally.
 
@@ -145,6 +145,36 @@ async def halt_response(request, response):
 	return text('I halted the response')
 ```
 
+### Blueprint Group Middleware
+Using this middleware will ensure that you can apply a common middleware to all the blueprints that form the
+current blueprint group under consideration.
+
+```python
+bp1 = Blueprint('bp1', url_prefix='/bp1')
+bp2 = Blueprint('bp2', url_prefix='/bp2')
+
+@bp1.middleware('request')
+async def bp1_only_middleware(request):
+    print('applied on Blueprint : bp1 Only')
+
+@bp1.route('/')
+async def bp1_route(request):
+    return text('bp1')
+
+@bp2.route('/<param>')
+async def bp2_route(request, param):
+    return text(param)
+
+group = Blueprint.group(bp1, bp2)
+
+@group.middleware('request')
+async def group_middleware(request):
+    print('common middleware applied for both bp1 and bp2')
+    
+# Register Blueprint group under the app
+app.blueprint(group)
+```
+
 ### Exceptions
 
 Exceptions can be applied exclusively to blueprints globally.
diff --git a/docs/sanic/request_data.md b/docs/sanic/request_data.md
index 01a182a3..065d8e13 100644
--- a/docs/sanic/request_data.md
+++ b/docs/sanic/request_data.md
@@ -19,6 +19,8 @@ The following variables are accessible as properties on `Request` objects:
   URL that resembles `?key1=value1&key2=value2`. If that URL were to be parsed,
   the `args` dictionary would look like `{'key1': ['value1'], 'key2': ['value2']}`.
   The request's `query_string` variable holds the unparsed string value.
+  Property is providing the default parsing strategy. If you would like to change it look to the section below
+  (`Changing the default parsing rules of the queryset`).
 
   ```python
   from sanic.response import json
@@ -28,9 +30,54 @@ The following variables are accessible as properties on `Request` objects:
       return json({ "parsed": True, "args": request.args, "url": request.url, "query_string": request.query_string })
   ```
 
-- `raw_args` (dict) - On many cases you would need to access the url arguments in
-  a less packed dictionary. For same previous URL `?key1=value1&key2=value2`, the
-  `raw_args` dictionary would look like `{'key1': 'value1', 'key2': 'value2'}`.
+- `query_args` (list) - On many cases you would need to access the url arguments in
+  a less packed form. `query_args` is the list of `(key, value)` tuples.
+  Property is providing the default parsing strategy. If you would like to change it look to the section below
+  (`Changing the default parsing rules of the queryset`).
+  For the same previous URL queryset `?key1=value1&key2=value2`, the
+  `query_args` list would look like `[('key1', 'value1'), ('key2', 'value2')]`.
+  And in case of the multiple params with the same key like `?key1=value1&key2=value2&key1=value3`
+  the `query_args` list would look like `[('key1', 'value1'), ('key2', 'value2'), ('key1', 'value3')]`.
+
+  The difference between Request.args and Request.query_args
+  for the queryset `?key1=value1&key2=value2&key1=value3`
+
+  ```python
+  from sanic import Sanic
+  from sanic.response import json
+
+  app = Sanic(__name__)
+
+
+  @app.route("/test_request_args")
+  async def test_request_args(request):
+      return json({
+          "parsed": True,
+          "url": request.url,
+          "query_string": request.query_string,
+          "args": request.args,
+          "raw_args": request.raw_args,
+          "query_args": request.query_args,
+      })
+
+  if __name__ == '__main__':
+      app.run(host="0.0.0.0", port=8000)
+  ```
+
+  Output
+
+  ```
+  {
+    "parsed":true,
+    "url":"http:\/\/0.0.0.0:8000\/test_request_args?key1=value1&key2=value2&key1=value3",
+    "query_string":"key1=value1&key2=value2&key1=value3",
+    "args":{"key1":["value1","value3"],"key2":["value2"]},
+    "raw_args":{"key1":"value1","key2":"value2"},
+    "query_args":[["key1","value1"],["key2","value2"],["key1","value3"]]
+  }
+  ```
+
+  `raw_args` contains only the first entry of `key1`. Will be deprecated in the future versions.
 
 - `files` (dictionary of `File` objects) - List of files that have a name, body, and type
 
@@ -106,6 +153,51 @@ The following variables are accessible as properties on `Request` objects:
 - `token`: The value of Authorization header: `Basic YWRtaW46YWRtaW4=`
 
 
+## Changing the default parsing rules of the queryset
+
+The default parameters that are using internally in `args` and `query_args` properties to parse queryset:
+
+- `keep_blank_values` (bool): `False` - flag indicating whether blank values in
+  percent-encoded queries should be treated as blank strings.
+  A true value indicates that blanks should be retained as blank
+  strings.  The default false value indicates that blank values
+  are to be ignored and treated as if they were  not included.
+- `strict_parsing` (bool): `False` - flag indicating what to do with parsing errors. If
+  false (the default), errors are silently ignored. If true,
+  errors raise a ValueError exception.
+- `encoding` and `errors` (str): 'utf-8' and 'replace' - specify how to decode percent-encoded sequences
+  into Unicode characters, as accepted by the bytes.decode() method.
+
+If you would like to change that default parameters you could call `get_args` and `get_query_args` methods
+with the new values.
+
+For the queryset `/?test1=value1&test2=&test3=value3`:
+
+```python
+from sanic.response import json
+
+@app.route("/query_string")
+def query_string(request):
+    args_with_blank_values = request.get_args(keep_blank_values=True)
+    return json({
+        "parsed": True,
+        "url": request.url,
+        "args_with_blank_values": args_with_blank_values,
+        "query_string": request.query_string
+    })
+```
+
+The output will be:
+
+```
+{
+    "parsed": true,
+    "url": "http:\/\/0.0.0.0:8000\/query_string?test1=value1&test2=&test3=value3",
+    "args_with_blank_values": {"test1": ["value1""], "test2": "", "test3": ["value3"]},
+    "query_string": "test1=value1&test2=&test3=value3"
+}
+```
+
 ## Accessing values using `get` and `getlist`
 
 The request properties which return a dictionary actually return a subclass of
diff --git a/sanic/app.py b/sanic/app.py
index e7ef89e0..a32d924c 100644
--- a/sanic/app.py
+++ b/sanic/app.py
@@ -15,6 +15,7 @@ from typing import Any, Optional, Type, Union
 from urllib.parse import urlencode, urlunparse
 
 from sanic import reloader_helpers
+from sanic.blueprint_group import BlueprintGroup
 from sanic.config import BASE_LOGO, Config
 from sanic.constants import HTTP_METHODS
 from sanic.exceptions import SanicException, ServerError, URLBuildError
@@ -597,9 +598,11 @@ class Sanic:
         :return: decorated method
         """
         if attach_to == "request":
-            self.request_middleware.append(middleware)
+            if middleware not in self.request_middleware:
+                self.request_middleware.append(middleware)
         if attach_to == "response":
-            self.response_middleware.appendleft(middleware)
+            if middleware not in self.response_middleware:
+                self.response_middleware.appendleft(middleware)
         return middleware
 
     # Decorator
@@ -681,7 +684,7 @@ class Sanic:
         :param options: option dictionary with blueprint defaults
         :return: Nothing
         """
-        if isinstance(blueprint, (list, tuple)):
+        if isinstance(blueprint, (list, tuple, BlueprintGroup)):
             for item in blueprint:
                 self.blueprint(item, **options)
             return
diff --git a/sanic/blueprint_group.py b/sanic/blueprint_group.py
new file mode 100644
index 00000000..9519d4f0
--- /dev/null
+++ b/sanic/blueprint_group.py
@@ -0,0 +1,120 @@
+from collections import MutableSequence
+
+
+class BlueprintGroup(MutableSequence):
+    """
+    This class provides a mechanism to implement a Blueprint Group
+    using the `Blueprint.group` method. To avoid having to re-write
+    some of the existing implementation, this class provides a custom
+    iterator implementation that will let you use the object of this
+    class as a list/tuple inside the existing implementation.
+    """
+
+    __slots__ = ("_blueprints", "_url_prefix")
+
+    def __init__(self, url_prefix=None):
+        """
+        Create a new Blueprint Group
+
+        :param url_prefix: URL: to be prefixed before all the Blueprint Prefix
+        """
+        self._blueprints = []
+        self._url_prefix = url_prefix
+
+    @property
+    def url_prefix(self):
+        """
+        Retrieve the URL prefix being used for the Current Blueprint Group
+        :return: string with url prefix
+        """
+        return self._url_prefix
+
+    @property
+    def blueprints(self):
+        """
+        Retrieve a list of all the available blueprints under this group.
+        :return: List of Blueprint instance
+        """
+        return self._blueprints
+
+    def __iter__(self):
+        """Tun the class Blueprint Group into an Iterable item"""
+        return iter(self._blueprints)
+
+    def __getitem__(self, item):
+        """
+        This method returns a blueprint inside the group specified by
+        an index value. This will enable indexing, splice and slicing
+        of the blueprint group like we can do with regular list/tuple.
+
+        This method is provided to ensure backward compatibility with
+        any of the pre-existing usage that might break.
+
+        :param item: Index of the Blueprint item in the group
+        :return: Blueprint object
+        """
+        return self._blueprints[item]
+
+    def __setitem__(self, index: int, item: object) -> None:
+        """
+        Abstract method implemented to turn the `BlueprintGroup` class
+        into a list like object to support all the existing behavior.
+
+        This method is used to perform the list's indexed setter operation.
+
+        :param index: Index to use for inserting a new Blueprint item
+        :param item: New `Blueprint` object.
+        :return: None
+        """
+        self._blueprints[index] = item
+
+    def __delitem__(self, index: int) -> None:
+        """
+        Abstract method implemented to turn the `BlueprintGroup` class
+        into a list like object to support all the existing behavior.
+
+        This method is used to delete an item from the list of blueprint
+        groups like it can be done on a regular list with index.
+
+        :param index: Index to use for removing a new Blueprint item
+        :return: None
+        """
+        del self._blueprints[index]
+
+    def __len__(self) -> int:
+        """
+        Get the Length of the blueprint group object.
+        :return: Length of Blueprint group object
+        """
+        return len(self._blueprints)
+
+    def insert(self, index: int, item: object) -> None:
+        """
+        The Abstract class `MutableSequence` leverages this insert method to
+        perform the `BlueprintGroup.append` operation.
+
+        :param index: Index to use for removing a new Blueprint item
+        :param item: New `Blueprint` object.
+        :return: None
+        """
+        self._blueprints.insert(index, item)
+
+    def middleware(self, *args, **kwargs):
+        """
+        A decorator that can be used to implement a Middleware plugin to
+        all of the Blueprints that belongs to this specific Blueprint Group.
+
+        In case of nested Blueprint Groups, the same middleware is applied
+        across each of the Blueprints recursively.
+
+        :param args: Optional positional Parameters to be use middleware
+        :param kwargs: Optional Keyword arg to use with Middleware
+        :return: Partial function to apply the middleware
+        """
+        kwargs["bp_group"] = True
+
+        def register_middleware_for_blueprints(fn):
+            for blueprint in self.blueprints:
+                blueprint.middleware(fn, *args, **kwargs)
+
+        return register_middleware_for_blueprints
diff --git a/sanic/blueprints.py b/sanic/blueprints.py
index f2575e57..c50de648 100644
--- a/sanic/blueprints.py
+++ b/sanic/blueprints.py
@@ -1,5 +1,6 @@
 from collections import defaultdict, namedtuple
 
+from sanic.blueprint_group import BlueprintGroup
 from sanic.constants import HTTP_METHODS
 from sanic.views import CompositionView
 
@@ -78,10 +79,12 @@ class Blueprint:
             for i in nested:
                 if isinstance(i, (list, tuple)):
                     yield from chain(i)
+                elif isinstance(i, BlueprintGroup):
+                    yield from i.blueprints
                 else:
                     yield i
 
-        bps = []
+        bps = BlueprintGroup(url_prefix=url_prefix)
         for bp in chain(blueprints):
             if bp.url_prefix is None:
                 bp.url_prefix = ""
@@ -327,7 +330,13 @@ class Blueprint:
             args = []
             return register_middleware(middleware)
         else:
-            return register_middleware
+            if kwargs.get("bp_group") and callable(args[0]):
+                middleware = args[0]
+                args = args[1:]
+                kwargs.pop("bp_group")
+                return register_middleware(middleware)
+            else:
+                return register_middleware
 
     def exception(self, *args, **kwargs):
         """
diff --git a/sanic/request.py b/sanic/request.py
index 90c9de19..76975801 100644
--- a/sanic/request.py
+++ b/sanic/request.py
@@ -2,11 +2,12 @@ import asyncio
 import email.utils
 import json
 import sys
+import warnings
 
 from cgi import parse_header
-from collections import namedtuple
+from collections import defaultdict, namedtuple
 from http.cookies import SimpleCookie
-from urllib.parse import parse_qs, unquote, urlunparse
+from urllib.parse import parse_qs, parse_qsl, unquote, urlunparse
 
 from httptools import parse_url
 
@@ -83,6 +84,7 @@ class Request(dict):
         "headers",
         "method",
         "parsed_args",
+        "parsed_not_grouped_args",
         "parsed_files",
         "parsed_form",
         "parsed_json",
@@ -109,7 +111,8 @@ class Request(dict):
         self.parsed_json = None
         self.parsed_form = None
         self.parsed_files = None
-        self.parsed_args = None
+        self.parsed_args = defaultdict(RequestParameters)
+        self.parsed_not_grouped_args = defaultdict(list)
         self.uri_template = None
         self._cookies = None
         self.stream = None
@@ -199,21 +202,117 @@ class Request(dict):
 
         return self.parsed_files
 
-    @property
-    def args(self):
-        if self.parsed_args is None:
+    def get_args(
+        self,
+        keep_blank_values: bool = False,
+        strict_parsing: bool = False,
+        encoding: str = "utf-8",
+        errors: str = "replace",
+    ) -> RequestParameters:
+        """
+        Method to parse `query_string` using `urllib.parse.parse_qs`.
+        This methods is used by `args` property.
+        Can be used directly if you need to change default parameters.
+        :param keep_blank_values: flag indicating whether blank values in
+            percent-encoded queries should be treated as blank strings.
+            A true value indicates that blanks should be retained as blank
+            strings.  The default false value indicates that blank values
+            are to be ignored and treated as if they were  not included.
+        :type keep_blank_values: bool
+        :param strict_parsing: flag indicating what to do with parsing errors.
+            If false (the default), errors are silently ignored. If true,
+            errors raise a ValueError exception.
+        :type strict_parsing: bool
+        :param encoding: specify how to decode percent-encoded sequences
+            into Unicode characters, as accepted by the bytes.decode() method.
+        :type encoding: str
+        :param errors: specify how to decode percent-encoded sequences
+            into Unicode characters, as accepted by the bytes.decode() method.
+        :type errors: str
+        :return: RequestParameters
+        """
+        if not self.parsed_args[
+            (keep_blank_values, strict_parsing, encoding, errors)
+        ]:
             if self.query_string:
-                self.parsed_args = RequestParameters(
-                    parse_qs(self.query_string)
+                self.parsed_args[
+                    (keep_blank_values, strict_parsing, encoding, errors)
+                ] = RequestParameters(
+                    parse_qs(
+                        qs=self.query_string,
+                        keep_blank_values=keep_blank_values,
+                        strict_parsing=strict_parsing,
+                        encoding=encoding,
+                        errors=errors,
+                    )
                 )
-            else:
-                self.parsed_args = RequestParameters()
-        return self.parsed_args
+
+        return self.parsed_args[
+            (keep_blank_values, strict_parsing, encoding, errors)
+        ]
+
+    args = property(get_args)
 
     @property
-    def raw_args(self):
+    def raw_args(self) -> dict:
+        if self.app.debug:  # pragma: no cover
+            warnings.simplefilter("default")
+        warnings.warn(
+            "Use of raw_args will be deprecated in "
+            "the future versions. Please use args or query_args "
+            "properties instead",
+            DeprecationWarning,
+        )
         return {k: v[0] for k, v in self.args.items()}
 
+    def get_query_args(
+        self,
+        keep_blank_values: bool = False,
+        strict_parsing: bool = False,
+        encoding: str = "utf-8",
+        errors: str = "replace",
+    ) -> list:
+        """
+        Method to parse `query_string` using `urllib.parse.parse_qsl`.
+        This methods is used by `query_args` property.
+        Can be used directly if you need to change default parameters.
+        :param keep_blank_values: flag indicating whether blank values in
+            percent-encoded queries should be treated as blank strings.
+            A true value indicates that blanks should be retained as blank
+            strings.  The default false value indicates that blank values
+            are to be ignored and treated as if they were  not included.
+        :type keep_blank_values: bool
+        :param strict_parsing: flag indicating what to do with parsing errors.
+            If false (the default), errors are silently ignored. If true,
+            errors raise a ValueError exception.
+        :type strict_parsing: bool
+        :param encoding: specify how to decode percent-encoded sequences
+            into Unicode characters, as accepted by the bytes.decode() method.
+        :type encoding: str
+        :param errors: specify how to decode percent-encoded sequences
+            into Unicode characters, as accepted by the bytes.decode() method.
+        :type errors: str
+        :return: list
+        """
+        if not self.parsed_not_grouped_args[
+            (keep_blank_values, strict_parsing, encoding, errors)
+        ]:
+            if self.query_string:
+                self.parsed_not_grouped_args[
+                    (keep_blank_values, strict_parsing, encoding, errors)
+                ] = parse_qsl(
+                    qs=self.query_string,
+                    keep_blank_values=keep_blank_values,
+                    strict_parsing=strict_parsing,
+                    encoding=encoding,
+                    errors=errors,
+                )
+        return self.parsed_not_grouped_args[
+            (keep_blank_values, strict_parsing, encoding, errors)
+        ]
+
+    query_args = property(get_query_args)
+
     @property
     def cookies(self):
         if self._cookies is None:


Test patch:
diff --git a/docs/sanic/testing.md b/docs/sanic/testing.md
index 00a604bf..28caac12 100644
--- a/docs/sanic/testing.md
+++ b/docs/sanic/testing.md
@@ -59,6 +59,23 @@ the available arguments to aiohttp can be found
 [in the documentation for ClientSession](https://aiohttp.readthedocs.io/en/stable/client_reference.html#client-session).
 
 
+## Using a random port
+
+If you need to test using a free unpriveleged port chosen by the kernel
+instead of the default with `SanicTestClient`, you can do so by specifying
+`port=None`. On most systems the port will be in the range 1024 to 65535.
+
+```python
+# Import the Sanic app, usually created with Sanic(__name__)
+from external_server import app
+from sanic.testing import SanicTestClient
+
+def test_index_returns_200():
+    request, response = SanicTestClient(app, port=None).get('/')
+    assert response.status == 200
+```
+
+
 ## pytest-sanic
 
 [pytest-sanic](https://github.com/yunstanford/pytest-sanic) is a pytest plugin, it helps you to test your code asynchronously.
diff --git a/sanic/testing.py b/sanic/testing.py
index 19f87095..2e2f60f0 100644
--- a/sanic/testing.py
+++ b/sanic/testing.py
@@ -3,6 +3,7 @@ from json import JSONDecodeError
 from sanic.exceptions import MethodNotSupported
 from sanic.log import logger
 from sanic.response import text
+from socket import socket
 
 
 HOST = "127.0.0.1"
@@ -11,19 +12,13 @@ PORT = 42101
 
 class SanicTestClient:
     def __init__(self, app, port=PORT):
+        """Use port=None to bind to a random port"""
         self.app = app
         self.port = port
 
-    async def _local_request(self, method, uri, cookies=None, *args, **kwargs):
+    async def _local_request(self, method, url, cookies=None, *args, **kwargs):
         import aiohttp
 
-        if uri.startswith(("http:", "https:", "ftp:", "ftps://" "//")):
-            url = uri
-        else:
-            url = "http://{host}:{port}{uri}".format(
-                host=HOST, port=self.port, uri=uri
-            )
-
         logger.info(url)
         conn = aiohttp.TCPConnector(ssl=False)
         async with aiohttp.ClientSession(
@@ -79,11 +74,27 @@ class SanicTestClient:
             else:
                 return self.app.error_handler.default(request, exception)
 
+        if self.port:
+            server_kwargs = dict(host=HOST, port=self.port, **server_kwargs)
+            host, port = HOST, self.port
+        else:
+            sock = socket()
+            sock.bind((HOST, 0))
+            server_kwargs = dict(sock=sock, **server_kwargs)
+            host, port = sock.getsockname()
+
+        if uri.startswith(("http:", "https:", "ftp:", "ftps://", "//")):
+            url = uri
+        else:
+            url = "http://{host}:{port}{uri}".format(
+                host=host, port=port, uri=uri
+            )
+
         @self.app.listener("after_server_start")
         async def _collect_response(sanic, loop):
             try:
                 response = await self._local_request(
-                    method, uri, *request_args, **request_kwargs
+                    method, url, *request_args, **request_kwargs
                 )
                 results[-1] = response
             except Exception as e:
@@ -91,7 +102,7 @@ class SanicTestClient:
                 exceptions.append(e)
             self.app.stop()
 
-        self.app.run(host=HOST, debug=debug, port=self.port, **server_kwargs)
+        self.app.run(debug=debug, **server_kwargs)
         self.app.listeners["after_server_start"].pop()
 
         if exceptions:
diff --git a/tests/test_blueprint_group.py b/tests/test_blueprint_group.py
new file mode 100644
index 00000000..fe1db7a3
--- /dev/null
+++ b/tests/test_blueprint_group.py
@@ -0,0 +1,180 @@
+from pytest import raises
+
+from sanic.app import Sanic
+from sanic.blueprints import Blueprint
+from sanic.request import Request
+from sanic.response import text, HTTPResponse
+
+MIDDLEWARE_INVOKE_COUNTER = {"request": 0, "response": 0}
+
+AUTH = "dGVzdDp0ZXN0Cg=="
+
+
+def test_bp_group_indexing(app: Sanic):
+    blueprint_1 = Blueprint("blueprint_1", url_prefix="/bp1")
+    blueprint_2 = Blueprint("blueprint_2", url_prefix="/bp2")
+
+    group = Blueprint.group(blueprint_1, blueprint_2)
+    assert group[0] == blueprint_1
+
+    with raises(expected_exception=IndexError) as e:
+        _ = group[3]
+
+
+def test_bp_group_with_additional_route_params(app: Sanic):
+    blueprint_1 = Blueprint("blueprint_1", url_prefix="/bp1")
+    blueprint_2 = Blueprint("blueprint_2", url_prefix="/bp2")
+
+    @blueprint_1.route(
+        "/request_path", methods=frozenset({"PUT", "POST"}), version=2
+    )
+    def blueprint_1_v2_method_with_put_and_post(request: Request):
+        if request.method == "PUT":
+            return text("PUT_OK")
+        elif request.method == "POST":
+            return text("POST_OK")
+
+    @blueprint_2.route(
+        "/route/<param>", methods=frozenset({"DELETE", "PATCH"}), name="test"
+    )
+    def blueprint_2_named_method(request: Request, param):
+        if request.method == "DELETE":
+            return text("DELETE_{}".format(param))
+        elif request.method == "PATCH":
+            return text("PATCH_{}".format(param))
+
+    blueprint_group = Blueprint.group(
+        blueprint_1, blueprint_2, url_prefix="/api"
+    )
+
+    @blueprint_group.middleware("request")
+    def authenticate_request(request: Request):
+        global AUTH
+        auth = request.headers.get("authorization")
+        if auth:
+            # Dummy auth check. We can have anything here and it's fine.
+            if AUTH not in auth:
+                return text("Unauthorized", status=401)
+        else:
+            return text("Unauthorized", status=401)
+
+    @blueprint_group.middleware("response")
+    def enhance_response_middleware(request: Request, response: HTTPResponse):
+        response.headers.add("x-test-middleware", "value")
+
+    app.blueprint(blueprint_group)
+
+    header = {"authorization": " ".join(["Basic", AUTH])}
+    _, response = app.test_client.put(
+        "/v2/api/bp1/request_path", headers=header
+    )
+    assert response.text == "PUT_OK"
+    assert response.headers.get("x-test-middleware") == "value"
+
+    _, response = app.test_client.post(
+        "/v2/api/bp1/request_path", headers=header
+    )
+    assert response.text == "POST_OK"
+
+    _, response = app.test_client.delete("/api/bp2/route/bp2", headers=header)
+    assert response.text == "DELETE_bp2"
+
+    _, response = app.test_client.patch("/api/bp2/route/bp2", headers=header)
+    assert response.text == "PATCH_bp2"
+
+    _, response = app.test_client.get("/v2/api/bp1/request_path")
+    assert response.status == 401
+
+
+def test_bp_group(app: Sanic):
+    blueprint_1 = Blueprint("blueprint_1", url_prefix="/bp1")
+    blueprint_2 = Blueprint("blueprint_2", url_prefix="/bp2")
+
+    @blueprint_1.route("/")
+    def blueprint_1_default_route(request):
+        return text("BP1_OK")
+
+    @blueprint_2.route("/")
+    def blueprint_2_default_route(request):
+        return text("BP2_OK")
+
+    blueprint_group_1 = Blueprint.group(
+        blueprint_1, blueprint_2, url_prefix="/bp"
+    )
+
+    blueprint_3 = Blueprint("blueprint_3", url_prefix="/bp3")
+
+    @blueprint_group_1.middleware("request")
+    def blueprint_group_1_middleware(request):
+        global MIDDLEWARE_INVOKE_COUNTER
+        MIDDLEWARE_INVOKE_COUNTER["request"] += 1
+
+    @blueprint_3.route("/")
+    def blueprint_3_default_route(request):
+        return text("BP3_OK")
+
+    blueprint_group_2 = Blueprint.group(
+        blueprint_group_1, blueprint_3, url_prefix="/api"
+    )
+
+    @blueprint_group_2.middleware("response")
+    def blueprint_group_2_middleware(request, response):
+        global MIDDLEWARE_INVOKE_COUNTER
+        MIDDLEWARE_INVOKE_COUNTER["response"] += 1
+
+    app.blueprint(blueprint_group_2)
+
+    @app.route("/")
+    def app_default_route(request):
+        return text("APP_OK")
+
+    _, response = app.test_client.get("/")
+    assert response.text == "APP_OK"
+
+    _, response = app.test_client.get("/api/bp/bp1")
+    assert response.text == "BP1_OK"
+
+    _, response = app.test_client.get("/api/bp/bp2")
+    assert response.text == "BP2_OK"
+
+    _, response = app.test_client.get("/api/bp3")
+    assert response.text == "BP3_OK"
+
+    assert MIDDLEWARE_INVOKE_COUNTER["response"] == 4
+    assert MIDDLEWARE_INVOKE_COUNTER["request"] == 4
+
+
+def test_bp_group_list_operations(app: Sanic):
+    blueprint_1 = Blueprint("blueprint_1", url_prefix="/bp1")
+    blueprint_2 = Blueprint("blueprint_2", url_prefix="/bp2")
+
+    @blueprint_1.route("/")
+    def blueprint_1_default_route(request):
+        return text("BP1_OK")
+
+    @blueprint_2.route("/")
+    def blueprint_2_default_route(request):
+        return text("BP2_OK")
+
+    blueprint_group_1 = Blueprint.group(
+        blueprint_1, blueprint_2, url_prefix="/bp"
+    )
+
+    blueprint_3 = Blueprint("blueprint_2", url_prefix="/bp3")
+
+    @blueprint_3.route("/second")
+    def blueprint_3_second_route(request):
+        return text("BP3_OK")
+
+    assert len(blueprint_group_1) == 2
+
+    blueprint_group_1.append(blueprint_3)
+    assert len(blueprint_group_1) == 3
+
+    del blueprint_group_1[2]
+    assert len(blueprint_group_1) == 2
+
+    blueprint_group_1[1] = blueprint_3
+    assert len(blueprint_group_1) == 2
+
+    assert blueprint_group_1.url_prefix == "/bp"
diff --git a/tests/test_requests.py b/tests/test_requests.py
index e10c4538..2f587361 100644
--- a/tests/test_requests.py
+++ b/tests/test_requests.py
@@ -10,7 +10,7 @@ import pytest
 from sanic import Sanic
 from sanic import Blueprint
 from sanic.exceptions import ServerError
-from sanic.request import DEFAULT_HTTP_CONTENT_TYPE
+from sanic.request import DEFAULT_HTTP_CONTENT_TYPE, RequestParameters
 from sanic.response import json, text
 from sanic.testing import HOST, PORT
 
@@ -130,6 +130,9 @@ def test_query_string(app):
 
     assert request.args.get("test1") == "1"
     assert request.args.get("test2") == "false"
+    assert request.args.getlist("test2") == ["false", "true"]
+    assert request.args.getlist("test1") == ["1"]
+    assert request.args.get("test3", default="My value") == "My value"
 
 
 def test_uri_template(app):
@@ -646,6 +649,77 @@ def test_request_raw_args(app):
     assert request.raw_args == params
 
 
+def test_request_query_args(app):
+    # test multiple params with the same key
+    params = [('test', 'value1'), ('test', 'value2')]
+
+    @app.get("/")
+    def handler(request):
+        return text("pass")
+
+    request, response = app.test_client.get("/", params=params)
+
+    assert request.query_args == params
+
+    # test cached value
+    assert request.parsed_not_grouped_args[(False, False, "utf-8", "replace")] == request.query_args
+
+    # test params directly in the url
+    request, response = app.test_client.get("/?test=value1&test=value2")
+
+    assert request.query_args == params
+
+    # test unique params
+    params = [('test1', 'value1'), ('test2', 'value2')]
+
+    request, response = app.test_client.get("/", params=params)
+
+    assert request.query_args == params
+
+    # test no params
+    request, response = app.test_client.get("/")
+
+    assert not request.query_args
+
+
+def test_request_query_args_custom_parsing(app):
+    @app.get("/")
+    def handler(request):
+        return text("pass")
+
+    request, response = app.test_client.get("/?test1=value1&test2=&test3=value3")
+
+    assert request.get_query_args(
+        keep_blank_values=True
+    ) == [
+        ('test1', 'value1'), ('test2', ''), ('test3', 'value3')
+    ]
+    assert request.query_args == [
+        ('test1', 'value1'), ('test3', 'value3')
+    ]
+    assert request.get_query_args(
+        keep_blank_values=False
+    ) == [
+        ('test1', 'value1'), ('test3', 'value3')
+    ]
+
+    assert request.get_args(
+        keep_blank_values=True
+    ) == RequestParameters(
+        {"test1": ["value1"], "test2": [""], "test3": ["value3"]}
+    )
+
+    assert request.args == RequestParameters(
+        {"test1": ["value1"], "test3": ["value3"]}
+    )
+
+    assert request.get_args(
+        keep_blank_values=False
+    ) == RequestParameters(
+        {"test1": ["value1"], "test3": ["value3"]}
+    )
+
+
 def test_request_cookies(app):
 
     cookies = {"test": "OK"}
diff --git a/tests/test_test_client_port.py b/tests/test_test_client_port.py
new file mode 100644
index 00000000..a49d9f81
--- /dev/null
+++ b/tests/test_test_client_port.py
@@ -0,0 +1,34 @@
+import socket
+
+from sanic.testing import PORT, SanicTestClient
+from sanic.response import json, text
+
+# ------------------------------------------------------------ #
+#  UTF-8
+# ------------------------------------------------------------ #
+
+
+def test_test_client_port_none(app):
+    @app.get('/get')
+    def handler(request):
+        return text('OK')
+
+    test_client = SanicTestClient(app, port=None)
+
+    request, response = test_client.get('/get')
+    assert response.text == 'OK'
+
+    request, response = test_client.post('/get')
+    assert response.status == 405
+
+
+def test_test_client_port_default(app):
+    @app.get('/get')
+    def handler(request):
+        return json(request.transport.get_extra_info('sockname')[1])
+
+    test_client = SanicTestClient(app)
+    assert test_client.port == PORT
+
+    request, response = test_client.get('/get')
+    assert response.json == PORT