Repository: burnash/gspread

Issue:
PROPOSAL: changes to `get_records`/`get_all_records`
`get_all_records` is a method which has provided much pain to maintain recently (issues #1363, #1355, #1354, #1352, #1294) (PRs #1364, #1357, #1345, #1343, #1301, #1028)

We propose that we simplify the method, and reduce its scope.

## New scope

Given a spreadsheet, `get_all_records` turns a header and following content into a list of records

<details>
<summary>API information</summary>

The API call will return to the pre-5.12 case:

https://github.com/burnash/gspread/blob/9cc862bc24f61de763944c59b2e1d270ef3b48cf/gspread/worksheet.py#L547

Even if the header is set to the 5th row, we still need to get all values, as we do not know the "final row" so we cannot use a row range like "5:999"

</details>

| my sheet | | |
| --- | --- | --- |
| fruit | price | |
| apple | 56 | |
| orange | 23 | |

```python
> get_all_records(header=2)
[
  {"fruit": "apple", "price": "56"},
  {"fruit": "orange", "price": "23"},
]
```

## Features to be changed

### Simplify the "duplicate headers" case

```diff
- get_all_records(expected_headers=["apple", "orange"])
+ get_all_records(allow_duplicate_headers=True)
```

##  Features to be removed

### `get_records` method, i.e., range_specification.

This means that headers + data will be obtained in a *single* API call. This simplifies a lot since we no longer have to pad the header/values to be the same length (#1357), as they will be the same length by nature of being from the same API call.

```diff
- get_records(first_index=10, last_index=15)
```

### `empty2zero` kwarg.

Its job can be done by existing `default_blank`

<details>
<summary>Details</summary>

Technically these differ currently. `default_blank` is used to replace a *single* blank header (not more), and *also* replace blank values. However, I think it should change to only replace values. Blank header should be left as-is and if there are multiple, duplicate headers can be ignored (see above).

</details>

```diff
- get_all_records(empty2zero=True)
+ get_all_records(default_blank=0)
```

## Features to be added

### `utils.to_records`

Add a utils function to turn a list of headers + values into a list of dictionaries

<details>
<summary>implementation details</summary>

this will basically be a wrapper of

https://github.com/burnash/gspread/blob/495a3290612755a433c06d7ce5a70b0271d0c695/gspread/worksheet.py#L757

</details>

```diff
+ utils.to_records(header, values)
```

This can be put into documentation to tell people how to make their own list of records, given a header and values.

```diff
+ Want to use get_records differently? You can make your own by using `utils.to_records`:
+ header = worksheet.get("4:4")
+ values = worksheet.get("20:40")
+ records = utils.to_records(header, values)
```



Discussion:

Patch:
diff --git a/gspread/utils.py b/gspread/utils.py
index 880991c..3e3a055 100644
--- a/gspread/utils.py
+++ b/gspread/utils.py
@@ -878,6 +878,16 @@ def get_a1_from_absolute_range(range_name: str) -> str:
     return range_name
 
 
+def to_records(
+    headers: Iterable[Any] = [], values: Iterable[Iterable[Any]] = [[]]
+) -> List[Dict[str, Union[str, int, float]]]:
+    """Builds the list of dictionaries, all of them have the headers sequence as keys set,
+    each key is associated to the corresponding value for the same index in the sub-list.
+    There are as many dictionaries as they are entry in the of given values."""
+
+    return [dict(zip(headers, row)) for row in values]
+
+
 # SHOULD NOT BE NEEDED UNTIL NEXT MAJOR VERSION
 # def deprecation_warning(version: str, msg: str) -> None:
 #     """Emit a deprecation warning.
diff --git a/gspread/worksheet.py b/gspread/worksheet.py
index 8b4f551..e2de7e3 100644
--- a/gspread/worksheet.py
+++ b/gspread/worksheet.py
@@ -55,6 +55,7 @@ from .utils import (
     is_full_a1_notation,
     numericise_all,
     rowcol_to_a1,
+    to_records,
 )
 
 CellFormat = TypedDict(
@@ -498,10 +499,10 @@ class Worksheet:
 
             # Read all rows from the sheet
             >>> worksheet.get_all_records()
-            {
+            [
                 {"A1": "A6", "B2": "B7", "C3": "C8"},
                 {"A1": "A11", "B2": "B12", "C3": "C13"}
-            }
+            ]
         """
         entire_sheet = self.get(
             value_render_option=value_render_option,
@@ -551,9 +552,7 @@ class Worksheet:
                 for row in values
             ]
 
-        formatted_records = [dict(zip(keys, row)) for row in values]
-
-        return formatted_records
+        return to_records(keys, values)
 
     def get_all_cells(self) -> List[Cell]:
         """Returns a list of all `Cell` of the current sheet."""


Test patch:
diff --git a/tests/utils_test.py b/tests/utils_test.py
index 14144fa..187e74e 100644
--- a/tests/utils_test.py
+++ b/tests/utils_test.py
@@ -465,3 +465,41 @@ class UtilsTest(unittest.TestCase):
         self.assertEqual(utils.get_a1_from_absolute_range("A1:B2"), "A1:B2")
         self.assertEqual(utils.get_a1_from_absolute_range("A1:B"), "A1:B")
         self.assertEqual(utils.get_a1_from_absolute_range("2"), "2")
+
+    def test_to_records_empty_args(self):
+        """Test to_records with empty args"""
+
+        self.assertListEqual(utils.to_records([], []), [])
+        self.assertListEqual(utils.to_records([], [[]]), [{}])
+        self.assertListEqual(utils.to_records(["a1", "b2"], []), [])
+        self.assertListEqual(utils.to_records(["a1", "b2"], [[]]), [{}])
+        self.assertListEqual(utils.to_records([], [["a1"]]), [{}])
+        self.assertListEqual(utils.to_records([], [["a1"], ["a2"]]), [{}, {}])
+        self.assertListEqual(utils.to_records([], [[], ["a2"]]), [{}, {}])
+
+    def test_to_records(self):
+        """Test to_records with values"""
+
+        headers = ["HA", "HB", "HC"]
+        values = [["A2", "B2", "C2"], ["A3", "B3"], ["", "B4", "C4"]]
+
+        records = utils.to_records(headers, values)
+
+        self.assertEqual(len(values), len(records))
+
+        for i in range(len(records)):
+            record = records[i]
+            keys = record.keys()
+
+            # Some rows have shorter values ("A3", "B3")
+            # so the list of keys is smaller
+            # but never bigger than the list of headers
+            self.assertLessEqual(len(keys), len(headers))
+
+            # Each resulting key must be part of the given header
+            for key in keys:
+                self.assertIn(key, headers)
+
+                # given key are unordered
+                # but they must match a value from the given input values
+                self.assertIn(record[key], values[i])