Repository: django-json-api/django-rest-framework-json-api

Issue:
`included` parameter potentially includes the object itself
Let's say we have a class `Room` and a class `Person` with a foreign key to `Room`. Now I want to create a new person in the room and reload all the people already in the room because adding the person might have changed them for some reason. Therefor I send a `POST` request containing the new person to 
```
persons/?include=room.persons`
```
This lead to the newly created person to be included twice, once in the `data` property of the reply and once in the `included` property. Even though this isn't explicitly forbidden by jsonapi.org, Ember doesn't like this. IMHO an object shouldn't be included twice in a reply - meaning that this should be fixed on the DJA side of things.


Discussion:
I think this is a tricky issue as it might be that different frameworks have different expectations. Question is what the specification defines in the end to see where it needs to be fixed.

Specification of discussion would be on [compound documents](https://jsonapi.org/format/#document-compound-documents) which sound a bit ambiguous on this issue.

This said do you have any other discussion links within the Ember or JSON API community where this issue has been discussed?
I didn't post this anywhere else. When I re-read the specification I thought it is pretty clear on this:

> A compound document MUST NOT include more than one resource object for each type and id pair.
> 
> 
> Note: In a single document, you can think of the type and id as a composite key that uniquely references resource objects in another part of the document.
>
> Note: This approach ensures that a single canonical resource object is returned with each response, even when the same resource is referenced multiple times.
> In a compound document, all included resources MUST be represented as an array of resource objects in a top-level included member.

Here the specification states that compound document is the top-level included member. This doesn't include the default resource object, so the statement that `A compound document MUST NOT include more than one resource` is as I understand it only relevant for the included member.

If it is understood this way how DJA implements it now would be OK.

I think in terms of payload fixing this would actually make sense, I am just not so sure on the clarity of the specification on this point, so it would be good to clarify it.
The crucial point is that it says in your quote *in a compound document*. It doesn't say that the included top level member is *the compound document* itself.

IMHO the specification is quite clear about what a compound document is:

> To reduce the number of HTTP requests, servers MAY allow responses that include related resources along with the requested primary resources. **Such responses are called “compound documents”.**

i.e. the full response, including the `data` and the `included` top level member are part of the response called a *compound document*.

However, I will open an issue on the JSON API side later today.
I have overread this part, but with this in mind I agree that a resource object should not be added twice including primary data.

I guess it never can hurt to double check though with JSON API. Marking this as bug and a Pull Request is very welcome.


I didn't have the time to create the issue at JSON API. I'll leave it at that and look into fixing the problem. Would you happen to have some pointers for me to where to start looking to fix this?
The resource itself should be part of the included_cache so it will be skipped in the renderer if it gets included again.

See https://github.com/django-json-api/django-rest-framework-json-api/blob/master/rest_framework_json_api/renderers.py#L383
I'm almost ready to submit the PR. Before I do a couple of questions:

* I wrote a integration test which tests this case but no unit test. Do we also need to unit-test this case?

* Adding the resource itself to the included_cache is IMHO not the best approach as we then have to removed it again before building the included attribute of the response. Therefor I decided to just check in the end whether it is in the cache and remove it in this case. Could this lead to any problems I didn't yet fully grasp?

Resolving patch:
diff --git a/rest_framework_json_api/renderers.py b/rest_framework_json_api/renderers.py
--- a/rest_framework_json_api/renderers.py
+++ b/rest_framework_json_api/renderers.py
@@ -643,6 +643,21 @@ def render(self, data, accepted_media_type=None, renderer_context=None):
         else:
             render_data['data'] = json_api_data
 
+        if included_cache:
+            if isinstance(json_api_data, list):
+                objects = json_api_data
+            else:
+                objects = [json_api_data]
+
+            for object in objects:
+                obj_type = object.get('type')
+                obj_id = object.get('id')
+                if obj_type in included_cache and \
+                   obj_id in included_cache[obj_type]:
+                    del included_cache[obj_type][obj_id]
+                if not included_cache[obj_type]:
+                    del included_cache[obj_type]
+
         if included_cache:
             render_data['included'] = list()
             for included_type in sorted(included_cache.keys()):


Test patch:
diff --git a/example/tests/conftest.py b/example/tests/conftest.py
--- a/example/tests/conftest.py
+++ b/example/tests/conftest.py
@@ -50,6 +50,13 @@ def multiple_entries(blog_factory, author_factory, entry_factory, comment_factor
     return entries
 
 
+@pytest.fixture
+def single_comment(blog, author, entry_factory, comment_factory):
+    entry = entry_factory(blog=blog, authors=(author,))
+    comment_factory(entry=entry)
+    return comment_factory(entry=entry)
+
+
 @pytest.fixture
 def single_company(art_project_factory, research_project_factory, company_factory):
     company = company_factory(future_projects=(research_project_factory(), art_project_factory()))
diff --git a/example/tests/integration/test_includes.py b/example/tests/integration/test_includes.py
--- a/example/tests/integration/test_includes.py
+++ b/example/tests/integration/test_includes.py
@@ -160,3 +160,22 @@ def test_deep_included_data_on_detail(single_entry, client):
     author_bio_count = len([resource for resource in included if resource["type"] == "authorBios"])
     expected_author_bio_count = single_entry.comments.filter(author__bio__isnull=False).count()
     assert author_bio_count == expected_author_bio_count, 'Detail author bio count is incorrect'
+
+
+def test_data_resource_not_included_again(single_comment, client):
+    # This test makes sure that the resource which is in the data field is excluded
+    # from the included field.
+    response = client.get(reverse("comment-detail", kwargs={'pk': single_comment.pk}) +
+                          '?include=entry.comments')
+
+    included = response.json().get('included')
+
+    included_comments = [resource for resource in included if resource["type"] == "comments"]
+    assert single_comment.pk not in [int(x.get('id')) for x in included_comments], \
+        "Resource of the data field duplicated in included"
+
+    comment_count = len(included_comments)
+    expected_comment_count = single_comment.entry.comments.count()
+    # The comment in the data attribute must not be included again.
+    expected_comment_count -= 1
+    assert comment_count == expected_comment_count, "Comment count incorrect"