Repository: encode/django-rest-framework

Issue:
Duplicate results of `ManyToManyField` when using `SearchFilter`.
I have two models:

<pre>
class Course(models.Model):
    name = models.CharField(max_length=255)

class Teacher(models.Model):
    name = models.CharField(max_length=255)
    courses = models.ManyToManyField('schools.Course')
</pre>

The serializer are:

<pre>
class CourseSerializer(serializers.ModelSerializer):
    teacher_set = serializers.PrimaryKeyRelatedField(many=True)
    class Meta:
        model = Course

class TeacherSerializer(serializers.ModelSerializer):
    class Meta:
        model = Teacher
</pre>

and the views for courses and teachers:

<pre>
class TeacherViewSet(ModelViewSet):
    serializer_class = TeacherSerializer
    queryset = Teacher.objects.all()
    filter_backends = (filters.DjangoFilterBackend,)
    filter_fields = ('courses',)

class CourseViewSet(StaffRequiredMixin, ModelViewSet):
    serializer_class = CourseSerializer
    queryset = Course.objects.all()
</pre>

After I create a course and a teacher, when I am adding the teacher to that course and do get list of teachers, it is returning me two exactly same teacher objects. 
I found that is because filter in the TeacherViewSet. If I remove filter, it is giving me correct result.  I was using django-filter version 0.7. When I changed to django-filter version 0.5.4. It is working fine then.
Is that a bug of version 0.7?

Edit:

What I did was: 
I created two courses (c1 and c2), then created a teacher (t1). Then I added t1 into c1. When do <code>GET /api/teachers</code>, I get t1 object, that's right. If add t1 into c2 and <code>GET /api/teachers</code> again, I will get two t1 objects.  

That won't happen if I put in OrderingFilter and set ordering

I am not a native English speaker, and the question I posted was not that clear. Hopefully this update can make more sense 



Discussion:
I found after I put ordering into TeacherViewSet, the duplicates gone. Is that I suppose to do ? 

Hi I'am having the same problem, downgraded django-filter to 0.5.4 now there's no duplicates results but filter_fields = ('field', 'field2') is not working.

@yuxuan Filtering through many to many can sometime be misleading. Did you filter on 2 or more courses when this happened ?

@xordoquy I updated the question. I didn't filter on 2 or more courses. I guess when add filter, the query will be base on the intermediate table. So if there is no filter parameters passed in, it will give duplicate results. I didn't read the source code, could be something else. 
@netoxico Is the updated situation same as you got?  There are some other issues in the old version. I think you would be better to use the latest version, and put in OrderingFilter as well, as what I did in the update question. That may solve your problem

This could do with a failing test case or example in order to help progress it.

DjangoCon sprint guidance - does the ticket make sense as expressed here? Can you replicate it? Can you write a failing test case for it? Does the fix look like it needs to be in django-filter or in REST framework? Are you able to submit a pull request with the fix?

Gonna try and write some failing tests for this.

So I just created this test case for this, but was not able to recreate.

https://gist.github.com/jpadilla/b493d98e9cd4ecf523bd

Any thoughts?

>  was not able to recreate

Okay, going to close this off then. Happy to reopen, but ideally would like someone to supply an example failing test case or confirm with more details on how to replicate.

I ran into this issue yesterday and have written a failing test case for it.

```
class AttributeModel(models.Model):
    label = models.CharField(max_length=32)


class SearchFilterModelM2M(models.Model):
    title = models.CharField(max_length=20)
    text = models.CharField(max_length=100)
    attributes = models.ManyToManyField(AttributeModel)


class SearchFilterM2MTests(TestCase):
    def setUp(self):
        # Sequence of title/text/attributes is:
        #
        # z   abc [1, 2, 3]
        # zz  bcd [1, 2, 3]
        # zzz cde [1, 2, 3]
        # ...
        for idx in range(3):
            label = 'w' * (idx + 1)
            AttributeModel(label=label)

        for idx in range(10):
            title = 'z' * (idx + 1)
            text = (
                chr(idx + ord('a')) +
                chr(idx + ord('b')) +
                chr(idx + ord('c'))
            )
            SearchFilterModelM2M(title=title, text=text).save()
        SearchFilterModelM2M.objects.get(title='zz').attributes.add(1, 2, 3)

    def test_m2m_search(self):
        class SearchListView(generics.ListAPIView):
            model = SearchFilterModelM2M
            filter_backends = (filters.SearchFilter,)
            search_fields = ('=title', 'text', 'attributes__label')

        view = SearchListView.as_view()
        request = factory.get('/', {'search': 'zz'})
        response = view(request)
        self.assertEqual(len(response.data), 1)
```

The simplest solution would be the inclusion of distinct() in [SearchFilter](https://github.com/tomchristie/django-rest-framework/blob/master/rest_framework/filters.py#L107), which makes this test pass.

This is in DRF 2.4.x

@mmedal thanks for the test case !

No prob! Let me know if you want a pull request for this one.

Can this issue be reopened?

If you could supply a pull request for a failing test case that'd be great, yes.

Actually, I think I already got this kind of things with pure Django application.
I won't have time to dig that until next week but it's likely an issue with django filters.

Retitled as this is an issue with `SearchFilter`, not `DjangoFilterBackend`


Resolving patch:
diff --git a/rest_framework/filters.py b/rest_framework/filters.py
--- a/rest_framework/filters.py
+++ b/rest_framework/filters.py
@@ -104,7 +104,7 @@ def filter_queryset(self, request, queryset, view):
         for search_term in self.get_search_terms(request):
             or_queries = [models.Q(**{orm_lookup: search_term})
                           for orm_lookup in orm_lookups]
-            queryset = queryset.filter(reduce(operator.or_, or_queries))
+            queryset = queryset.filter(reduce(operator.or_, or_queries)).distinct()
 
         return queryset
 


Test patch:
diff --git a/tests/test_filters.py b/tests/test_filters.py
--- a/tests/test_filters.py
+++ b/tests/test_filters.py
@@ -429,6 +429,56 @@ class SearchListView(generics.ListAPIView):
         reload_module(filters)
 
 
+class AttributeModel(models.Model):
+    label = models.CharField(max_length=32)
+
+
+class SearchFilterModelM2M(models.Model):
+    title = models.CharField(max_length=20)
+    text = models.CharField(max_length=100)
+    attributes = models.ManyToManyField(AttributeModel)
+
+
+class SearchFilterM2MSerializer(serializers.ModelSerializer):
+    class Meta:
+        model = SearchFilterModelM2M
+
+
+class SearchFilterM2MTests(TestCase):
+    def setUp(self):
+        # Sequence of title/text/attributes is:
+        #
+        # z   abc [1, 2, 3]
+        # zz  bcd [1, 2, 3]
+        # zzz cde [1, 2, 3]
+        # ...
+        for idx in range(3):
+            label = 'w' * (idx + 1)
+            AttributeModel(label=label)
+
+        for idx in range(10):
+            title = 'z' * (idx + 1)
+            text = (
+                chr(idx + ord('a')) +
+                chr(idx + ord('b')) +
+                chr(idx + ord('c'))
+            )
+            SearchFilterModelM2M(title=title, text=text).save()
+        SearchFilterModelM2M.objects.get(title='zz').attributes.add(1, 2, 3)
+
+    def test_m2m_search(self):
+        class SearchListView(generics.ListAPIView):
+            queryset = SearchFilterModelM2M.objects.all()
+            serializer_class = SearchFilterM2MSerializer
+            filter_backends = (filters.SearchFilter,)
+            search_fields = ('=title', 'text', 'attributes__label')
+
+        view = SearchListView.as_view()
+        request = factory.get('/', {'search': 'zz'})
+        response = view(request)
+        self.assertEqual(len(response.data), 1)
+
+
 class OrderingFilterModel(models.Model):
     title = models.CharField(max_length=20)
     text = models.CharField(max_length=100)