Repository: boto/boto

Issue:
DynamoDB2: Table.query() reverse parameter is misleading
While the documentation doesn't explicitly specify what the normal ordering of the results is, the actual operation is counterintuitive: the "reverse" parameter is mapped to the ScanIndexForward parameter of the underlying Amazon DynamoDB API.  Also the default value doesn't match documentation (False vs. None).  In other words, if I call the query method without specifying "reverse" or set it to False, ScanIndexForward will be false, and I get results in descending order.  If I set "reverse" to True, ScanIndexFoward will be true, and I get results in ascending order.

Since the default order is descending, simply deprecating the "reverse" parameter and adding a new one with opposite meaning doesn't fix the situation completely.  My suggestion is to deprecate the query method and add a new one (e.g. query2) with a parameter which maps directly to ScanIndexForward (e.g. "forward") or inversely (e.g. "reversed" or "descending").



Discussion:
Just ran into this same issue as well. Observed behavior is:
- Don't specify parameter - default is `reverse=False`, although [documentation](https://github.com/boto/boto/blob/develop/boto/dynamodb2/table.py#L826) erroneously says it will be set to `None`. This causes `ScanIndexForward=false` which returns the results in descending order (see AWS documentation [here](http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html#DDB-Query-request-ScanIndexForward)). This is counterintuitive. If I don't specify reverse, I expect results in the default, ascending order.
- Set `reverse=None`. This omits `ScanIndexForward` from the API call, which returns results in the default, ascending order. Behavior as expected.
- Set `reverse=True`. This sets `ScanIndexForward=true` which returns results in ascending order. This is wrong. If I specify reverse, I expect results in descending order.
- Set `reverse=False`. This sets `ScanIndexForward=false` which returns results in descending order, the opposite of what I expect. This is also wrong.

In the previous version of the boto dynamo API, the `table.query()` method took a [scan_index_forward parameter](https://github.com/boto/boto/blob/develop/boto/dynamodb/table.py#L425) which is consistent with the underlying DynamoDB API. Using the `reverse` argument is easier to understand, if it works as expected.

One possible solution would be to just invert the parameter's value as specified in [_query()](https://github.com/boto/boto/blob/develop/boto/dynamodb2/table.py#L973), something like:

``` python
    scan_index_forward = not reverse if reverse is not None else None
    kwargs = {
        ...
        'scan_index_forward': scan_index_forward,
        ...
    }
```

ran into this today, The document needs to be updated.


Resolving patch:
diff --git a/boto/dynamodb2/table.py b/boto/dynamodb2/table.py
--- a/boto/dynamodb2/table.py
+++ b/boto/dynamodb2/table.py
@@ -824,7 +824,7 @@ def query(self, limit=None, index=None, reverse=False, consistent=False,
         (Default: ``None``)
 
         Optionally accepts a ``reverse`` parameter, which will present the
-        results in reverse order. (Default: ``None`` - normal order)
+        results in reverse order. (Default: ``False`` - normal order)
 
         Optionally accepts a ``consistent`` parameter, which should be a
         boolean. If you provide ``True``, it will force a consistent read of
@@ -975,12 +975,14 @@ def _query(self, limit=None, index=None, reverse=False, consistent=False,
         kwargs = {
             'limit': limit,
             'index_name': index,
-            'scan_index_forward': reverse,
             'consistent_read': consistent,
             'select': select,
-            'attributes_to_get': attributes_to_get
+            'attributes_to_get': attributes_to_get,
         }
 
+        if reverse:
+            kwargs['scan_index_forward'] = False
+
         if exclusive_start_key:
             kwargs['exclusive_start_key'] = {}
 


Test patch:
diff --git a/tests/integration/dynamodb2/test_highlevel.py b/tests/integration/dynamodb2/test_highlevel.py
--- a/tests/integration/dynamodb2/test_highlevel.py
+++ b/tests/integration/dynamodb2/test_highlevel.py
@@ -515,4 +515,84 @@ def test_query_with_limits(self):
             [post['posted_by'] for post in all_posts],
             ['joe', 'jane', 'joe', 'joe', 'jane', 'joe']
         )
-        self.assertEqual(results._fetches, 3)
+        self.assertTrue(results._fetches >= 3)
+
+    def test_query_with_reverse(self):
+        posts = Table.create('more-posts', schema=[
+            HashKey('thread'),
+            RangeKey('posted_on')
+        ], throughput={
+            'read': 5,
+            'write': 5,
+        })
+        self.addCleanup(posts.delete)
+
+        # Wait for it.
+        time.sleep(60)
+
+        # Add some data.
+        test_data_path = os.path.join(
+            os.path.dirname(__file__),
+            'forum_test_data.json'
+        )
+        with open(test_data_path, 'r') as test_data:
+            data = json.load(test_data)
+
+            with posts.batch_write() as batch:
+                for post in data:
+                    batch.put_item(post)
+
+        time.sleep(5)
+
+        # Test the default order (ascending).
+        results = posts.query(
+            thread__eq='Favorite chiptune band?',
+            posted_on__gte='2013-12-24T00:00:00'
+        )
+        self.assertEqual(
+            [post['posted_on'] for post in results],
+            [
+                '2013-12-24T12:30:54',
+                '2013-12-24T12:35:40',
+                '2013-12-24T13:45:30',
+                '2013-12-24T14:15:14',
+                '2013-12-24T14:25:33',
+                '2013-12-24T15:22:22',
+            ]
+        )
+
+        # Test the explicit ascending order.
+        results = posts.query(
+            thread__eq='Favorite chiptune band?',
+            posted_on__gte='2013-12-24T00:00:00',
+            reverse=False
+        )
+        self.assertEqual(
+            [post['posted_on'] for post in results],
+            [
+                '2013-12-24T12:30:54',
+                '2013-12-24T12:35:40',
+                '2013-12-24T13:45:30',
+                '2013-12-24T14:15:14',
+                '2013-12-24T14:25:33',
+                '2013-12-24T15:22:22',
+            ]
+        )
+
+        # Test the explicit descending order.
+        results = posts.query(
+            thread__eq='Favorite chiptune band?',
+            posted_on__gte='2013-12-24T00:00:00',
+            reverse=True
+        )
+        self.assertEqual(
+            [post['posted_on'] for post in results],
+            [
+                '2013-12-24T15:22:22',
+                '2013-12-24T14:25:33',
+                '2013-12-24T14:15:14',
+                '2013-12-24T13:45:30',
+                '2013-12-24T12:35:40',
+                '2013-12-24T12:30:54',
+            ]
+        )
diff --git a/tests/unit/dynamodb2/test_table.py b/tests/unit/dynamodb2/test_table.py
--- a/tests/unit/dynamodb2/test_table.py
+++ b/tests/unit/dynamodb2/test_table.py
@@ -883,7 +883,7 @@ def test_max_page_size_and_bigger_limit_fetch_more(self):
         self.results.fetch_more()
         self.result_function.assert_called_with('john', greeting='Hello', limit=10)
         self.result_function.reset_mock()
-    
+
     def test_fetch_more(self):
         # First "page".
         self.results.fetch_more()
@@ -2149,7 +2149,7 @@ def test_private_query(self):
 
         mock_query.assert_called_once_with('users',
             consistent_read=False,
-            scan_index_forward=True,
+            scan_index_forward=False,
             index_name=None,
             attributes_to_get=None,
             limit=4,
@@ -2196,7 +2196,7 @@ def test_private_query(self):
             },
             index_name=None,
             attributes_to_get=None,
-            scan_index_forward=True,
+            scan_index_forward=False,
             limit=4,
             exclusive_start_key={
                 'username': {