Repository: Kinto/kinto

Issue:
Postgresql storage: id (and last_modified) are stored in the data column
We could «save» some storage space by avoiding this. It is rather trivial to do: strip record of those fields before storing it as `data`.

Current insert query is:

``` sql
            WITH delete_potential_tombstone AS (
                DELETE FROM deleted
                 WHERE id = '861756c7-9405-4819-a2cb-336606d3540e'
                   AND parent_id = '1234'
                   AND collection_id = 'test'
            )
            INSERT INTO records (id, parent_id, collection_id, data, last_modified)
            VALUES ('861756c7-9405-4819-a2cb-336606d3540e', '1234',
                    'test', ('{"code":{"sub":"c"},"id":"861756c7-9405-4819-a2cb-336606d3540e"}')::JSONB,
                    from_epoch(NULL))
            RETURNING id, as_epoch(last_modified) AS last_modified;

```

Postgresql storage: id (and last_modified) are stored in the data column
We could «save» some storage space by avoiding this. It is rather trivial to do: strip record of those fields before storing it as `data`.

Current insert query is:

``` sql
            WITH delete_potential_tombstone AS (
                DELETE FROM deleted
                 WHERE id = '861756c7-9405-4819-a2cb-336606d3540e'
                   AND parent_id = '1234'
                   AND collection_id = 'test'
            )
            INSERT INTO records (id, parent_id, collection_id, data, last_modified)
            VALUES ('861756c7-9405-4819-a2cb-336606d3540e', '1234',
                    'test', ('{"code":{"sub":"c"},"id":"861756c7-9405-4819-a2cb-336606d3540e"}')::JSONB,
                    from_epoch(NULL))
            RETURNING id, as_epoch(last_modified) AS last_modified;

```



Discussion:
@leplatrem do we need to remove `last_modified` field from `records` table?

Nope, just pop it out of the dict that is stored in the data column!

@leplatrem do we need to remove `last_modified` field from `records` table?

Nope, just pop it out of the dict that is stored in the data column!


Resolving patch:
diff --git a/kinto/core/storage/postgresql/__init__.py b/kinto/core/storage/postgresql/__init__.py
--- a/kinto/core/storage/postgresql/__init__.py
+++ b/kinto/core/storage/postgresql/__init__.py
@@ -230,6 +230,11 @@ def create(self, collection_id, parent_id, record, id_generator=None,
         else:
             record[id_field] = id_generator()
 
+        # Remove redundancy in data field
+        query_record = record.copy()
+        query_record.pop(id_field, None)
+        query_record.pop(modified_field, None)
+
         query = """
         WITH delete_potential_tombstone AS (
             DELETE FROM deleted
@@ -247,7 +252,7 @@ def create(self, collection_id, parent_id, record, id_generator=None,
                             parent_id=parent_id,
                             collection_id=collection_id,
                             last_modified=record.get(modified_field),
-                            data=json.dumps(record))
+                            data=json.dumps(query_record))
         with self.client.connect() as conn:
             result = conn.execute(query, placeholders)
             inserted = result.fetchone()
@@ -285,6 +290,12 @@ def update(self, collection_id, parent_id, object_id, record,
                id_field=DEFAULT_ID_FIELD,
                modified_field=DEFAULT_MODIFIED_FIELD,
                auth=None):
+
+        # Remove redundancy in data field
+        query_record = record.copy()
+        query_record.pop(id_field, None)
+        query_record.pop(modified_field, None)
+
         query_create = """
         WITH delete_potential_tombstone AS (
             DELETE FROM deleted
@@ -311,7 +322,7 @@ def update(self, collection_id, parent_id, object_id, record,
                             parent_id=parent_id,
                             collection_id=collection_id,
                             last_modified=record.get(modified_field),
-                            data=json.dumps(record))
+                            data=json.dumps(query_record))
 
         record = record.copy()
         record[id_field] = object_id


Test patch: