Repository: elastic/rally

Issue:
max_connections value is misleadingly logged prior to being overridden 

<!--
If you are filing a feature request, please remove the above bug
report block and provide responses for all of the below items.
-->

**Describe the feature**:
Currently `esrally` logs the output of  the `client_options` parameter at the time an [Elasticsearch client](https://github.com/elastic/rally/blob/master/esrally/client.py#L125) is created, for example:
```
2021-06-07 02:17:11,408 ActorAddr-(T|:62446)/PID:42037 esrally.client INFO Creating ES client connected to [{'host': '127.0.0.1', 'port': 39200}] with options [{'timeout': 180, 'max_connections': 2}]
```

The value displayed above for the `max_connections` is actually misleading. In the above log line, `max_connections` is equal to the number of simulated clients allocated per available core, yet this value is actually overridden later by [`AIOHttpConnection`](https://github.com/elastic/rally/blob/master/esrally/async_connection.py#L171) to a minimum of 256 per Elasticsearch client.
```
maxsize=max(256, kwargs.get("max_connections", 0)),
```

[The documentation is correct](https://esrally.readthedocs.io/en/stable/command_line_reference.html?highlight=max_connection#client-options) in explaining that the `max_connections` value is equal to the number of simulated clients, but _at least_ 256 connections. 

Therefore it is misleading for users to see this value being logged _before_ it is overridden.

Discussion:

Patch:
diff --git a/esrally/async_connection.py b/esrally/async_connection.py
index 3609dce9..a236a96c 100644
--- a/esrally/async_connection.py
+++ b/esrally/async_connection.py
@@ -168,7 +168,7 @@ class AIOHttpConnection(elasticsearch.AIOHttpConnection):
                          ssl_assert_fingerprint=ssl_assert_fingerprint,
                          # provided to the base class via `maxsize` to keep base class state consistent despite Rally
                          # calling the attribute differently.
-                         maxsize=max(256, kwargs.get("max_connections", 0)),
+                         maxsize=kwargs.get("max_connections", 0),
                          headers=headers,
                          ssl_context=ssl_context,
                          http_compress=http_compress,
diff --git a/esrally/utils/opts.py b/esrally/utils/opts.py
index a24bc1fb..49517e5b 100644
--- a/esrally/utils/opts.py
+++ b/esrally/utils/opts.py
@@ -211,7 +211,6 @@ class ClientOptions(ConnectOptions):
         final_client_options = {}
         for cluster, original_opts in self.all_client_options.items():
             amended_opts = dict(original_opts)
-            if "max_connections" not in amended_opts:
-                amended_opts["max_connections"] = max_connections
+            amended_opts["max_connections"] = max(256, amended_opts.get("max_connections", max_connections))
             final_client_options[cluster] = amended_opts
         return final_client_options


Test patch:
diff --git a/tests/utils/opts_test.py b/tests/utils/opts_test.py
index 67cd5983..fefba561 100644
--- a/tests/utils/opts_test.py
+++ b/tests/utils/opts_test.py
@@ -276,16 +276,23 @@ class TestClientOptions(TestCase):
             opts.ClientOptions(client_options_string,
                                  target_hosts=target_hosts).default)
 
-    def test_amends_with_max_connections(self):
+    def test_default_client_ops_with_max_connections(self):
         client_options_string = opts.ClientOptions.DEFAULT_CLIENT_OPTIONS
         target_hosts = opts.TargetHosts('{"default": ["10.17.0.5:9200"], "remote": ["88.33.22.15:9200"]}')
         self.assertEqual(
-            {"default": {"timeout": 60, "max_connections": 128}, "remote": {"timeout": 60, "max_connections": 128}},
-            opts.ClientOptions(client_options_string, target_hosts=target_hosts).with_max_connections(128))
+            {"default": {"timeout": 60, "max_connections": 256}, "remote": {"timeout": 60, "max_connections": 256}},
+            opts.ClientOptions(client_options_string, target_hosts=target_hosts).with_max_connections(256))
 
-    def test_keeps_already_specified_max_connections(self):
+    def test_sets_minimum_max_connections(self):
         client_options_string = '{"default": {"timeout":60,"max_connections":5}, "remote": {"timeout":60}}'
         target_hosts = opts.TargetHosts('{"default": ["10.17.0.5:9200"], "remote": ["88.33.22.15:9200"]}')
         self.assertEqual(
-            {"default": {"timeout": 60, "max_connections": 5}, "remote": {"timeout": 60, "max_connections": 32}},
+            {"default": {"timeout": 60, "max_connections": 256}, "remote": {"timeout": 60, "max_connections": 256}},
+            opts.ClientOptions(client_options_string, target_hosts=target_hosts).with_max_connections(5))
+
+    def test_keeps_greater_than_minimum_max_connections(self):
+        client_options_string = '{"default": {"timeout":60,"max_connections":512}, "remote": {"timeout":60,"max_connections":1024}}'
+        target_hosts = opts.TargetHosts('{"default": ["10.17.0.5:9200"], "remote": ["88.33.22.15:9200"]}')
+        self.assertEqual(
+            {"default": {"timeout": 60, "max_connections": 512}, "remote": {"timeout": 60, "max_connections": 1024}},
             opts.ClientOptions(client_options_string, target_hosts=target_hosts).with_max_connections(32))