Repository: carltongibson/django-filter

Issue:
Add "Any" filter to ChoicesField
The ChoicesField currently doesn't provide an "Any" or "Unknown" filter entry, so as soon as a ChoiceField is added to a search form, the user has to always select an entry.

I currently workaround this by adding an empty filter in the Filter's constructor, but it would be great if there could be an easier way to achieve this:

```
class NicerFilterSet(django_filters.FilterSet):

def __init__(self, *args, **kwargs):
    super(NicerFilterSet, self).__init__(*args, **kwargs)

    for name, field in self.filters.iteritems():
        if isinstance(field, ChoiceFilter):
            # Add "Any" entry to choice fields.
            field.extra['choices'] = tuple([("", "Any"), ] + list(field.extra['choices']))`
```

Add Any & None Handling to ChoiceFilter (and subclasses)
ChoiceFilter does not provide an empty choice to skip filtering on the field. 

Users can work around by adding the extra choice in `__init__`...

```
def __init__(self, *args, **kwargs):
    super(MyFilterSet, self).__init__(*args, **kwargs)
    self.filters['a_filter_field'].field.choices.insert(0, ('', u'---------'))
```

(Example adapted from #36)

... but it would be nice to have API to do this declaratively. 

As [pointed out in the discussion on #36](https://github.com/alex/django-filter/issues/36#issuecomment-4350491) we can't just automatically insert the extra choices since only certain widgets require it. 

An additional requirement (#214) is to filter for None values — i.e. "Any" means "Don't filter" where here we filter but only for None values. 

An overly simple ChoiceFilter subclass demonstrating **both** of these behaviours filtering against a related `Type` field looks like this: 

```
class TypeFilter(django_filters.ChoiceFilter):
    @property
    def field(self):
        if not hasattr(self, '_field'):

                choices = [
                    ('ANY',  'Anywhere'),('NONE', 'None'),
                ]

                type_qs = self.extra["queryset"]

                # This is coupled to the specific model
                choices += [('Type', [(t.pk, t.title) for t in type_qs],)]

                self._field = forms.ChoiceField(choices=choices, required=self.required, label=self.label, help_text='')

        return self._field

    def filter(self, qs, value):
        if value == '':
            return qs
        if value == 'ANY':
            return qs
        elif value == 'NONE':
            return qs.filter(type_id=None)
        else:
            return qs.filter(type_id=value)
```

Here we add the additional choices when generating the field and look out for them when filtering. We're not dealing with excludes, or distinct, or lookup type, or... 

Goals: 
- [ ] Document this better so users can see that implementing their own behaviour here is simple enough. 
- [ ] Work this into ChoiceFilter with a decent API.



Discussion:
Nice -- thanks for the workaround. Agreed that it's a pretty common requirement for a search/filter form.

Could you guys rework that into a pull request with docs and tests? Thx!

@57even commented in #172. I'm closing that and pasting it here to keep everything in one place:

> I think #45 is a great improvement but empty values gets the option "None" which is equivalent to no filter condition (instead of empty).
> 
> How could this be solved? I think the empty values should be completely ignored in AllValuesFilter.

Related to #36

Closing this in favour of #261

Here's a Python3 compatible version of the original work-around by @cvk77 (see top of thread)

```
class NicerFilterSet(django_filters.FilterSet):

def __init__(self, *args, **kwargs):
  super(NicerFilterSet, self).__init__(*args, **kwargs)

  for name, field in self.filters.items():
    if isinstance(field, ChoiceFilter):
      # Add "Any" entry to choice fields.
      field.extra['choices'] = tuple([("", "Any"), ] + list(field.extra['choices']))
```

Changes:
`self.filters.iteritems()` was changed to `self.filters.items()`

@carltongibson This is very close to what I was referring to in #252, but I think what I really want is an AllModelsFilter analogous to the AllValuesFilter, but for a ForiegnKey field. I am having trouble figuring out how to extend your example here to adapt to the model filter type since each option is simply a model instance and a None choice of any type (string or None) does not qualify as a queryset. I think this should possibly be a separate issue. My issue is certainly related to this issue, but I think the addition of an AllModelsFilter is enough to warrant a separate issue.

@marcus1487, I'm not understanding your issue. You want something like `qs.filter(fk=None)` right? (That gets you the QuerySet you're after yes?) What can't you do here? Can you put together a test case showing the behaviour you expect but can't get?

My issue is that I have to instantiate a Model type filter with a QuerySet. 

For example, let's say I have a Book model with a ForiegnKey field Author which can be null. Then when I set up a filter of books for authors I have to pass a QuerySet composed of Author objects. If I compose this from the `Authors.objects.all()` method then there is obviously no null option since Null/None is not a member of the authors table. If I compose the queryset from the foriegn keys of the books table I cannot get the author model objects. 

Maybe this is my problem. Is there a way to get the author models as a queryset from a query of the available author objects through the Books model. `.objects.values('author')` just gives me a dictionary which does not allow any Model `field_class` to have access to the correct objects. I may be missing something pretty straightforward, but I am getting really confused about what to do here.

Hi @marcus1487 — from your explanation I'm not sure what issue it is you're facing. It seems like a usage question though. 

If you need help can I ask you to send an email to the [Mailing List](https://groups.google.com/forum/#!forum/django-filter)? Therein please give a **code** example of where you're getting stuck and I'll see if I can say something sensible. Thanks. 


Resolving patch:
diff --git a/django_filters/conf.py b/django_filters/conf.py
--- a/django_filters/conf.py
+++ b/django_filters/conf.py
@@ -10,6 +10,12 @@
     'DISABLE_HELP_TEXT': False,
     'HELP_TEXT_FILTER': True,
     'HELP_TEXT_EXCLUDE': True,
+
+    # empty/null choices
+    'EMPTY_CHOICE_LABEL': '---------',
+    'NULL_CHOICE_LABEL': None,
+    'NULL_CHOICE_VALUE': 'null',
+
     'VERBOSE_LOOKUPS': {
         # transforms don't need to be verbose, since their expressions are chained
         'date': _('date'),
diff --git a/django_filters/filters.py b/django_filters/filters.py
--- a/django_filters/filters.py
+++ b/django_filters/filters.py
@@ -209,6 +209,39 @@ class BooleanFilter(Filter):
 class ChoiceFilter(Filter):
     field_class = forms.ChoiceField
 
+    def __init__(self, *args, **kwargs):
+        empty_label = kwargs.pop('empty_label', settings.EMPTY_CHOICE_LABEL)
+        null_label = kwargs.pop('null_label', settings.NULL_CHOICE_LABEL)
+        null_value = kwargs.pop('null_value', settings.NULL_CHOICE_VALUE)
+
+        self.null_value = null_value
+
+        if 'choices' in kwargs:
+            choices = kwargs.get('choices')
+
+            # coerce choices to list
+            if callable(choices):
+                choices = choices()
+            choices = list(choices)
+
+            # create the empty/null choices that prepend the original choices
+            prepend = []
+            if empty_label is not None:
+                prepend.append(('', empty_label))
+            if null_label is not None:
+                prepend.append((null_value, null_label))
+
+            kwargs['choices'] = prepend + choices
+
+        super(ChoiceFilter, self).__init__(*args, **kwargs)
+
+    def filter(self, qs, value):
+        if value != self.null_value:
+            return super(ChoiceFilter, self).filter(qs, value)
+
+        qs = self.get_method(qs)(**{'%s__%s' % (self.name, self.lookup_expr): None})
+        return qs.distinct() if self.distinct else qs
+
 
 class TypedChoiceFilter(Filter):
     field_class = forms.TypedChoiceField
@@ -454,6 +487,10 @@ class DateRangeFilter(ChoiceFilter):
     def __init__(self, *args, **kwargs):
         kwargs['choices'] = [
             (key, value[0]) for key, value in six.iteritems(self.options)]
+
+        # empty/null choices not relevant
+        kwargs.setdefault('empty_label', None)
+        kwargs.setdefault('null_label', None)
         super(DateRangeFilter, self).__init__(*args, **kwargs)
 
     def filter(self, qs, value):
@@ -602,6 +639,7 @@ def __init__(self, *args, **kwargs):
             kwargs['choices'] = self.build_choices(fields, field_labels)
 
         kwargs.setdefault('label', _('Ordering'))
+        kwargs.setdefault('null_label', None)
         super(OrderingFilter, self).__init__(*args, **kwargs)
 
     def get_ordering_value(self, param):
diff --git a/django_filters/filterset.py b/django_filters/filterset.py
--- a/django_filters/filterset.py
+++ b/django_filters/filterset.py
@@ -384,7 +384,7 @@ def display_text(name, fltr):
             if isinstance(opts.order_by[0], (list, tuple)):
                 choices = [(f[0], f[1]) for f in opts.order_by]
                 fields = {filters.get(f[0].lstrip('-')).name: f[0] for f in opts.order_by}
-                return OrderingFilter(choices=choices, fields=fields)
+                return OrderingFilter(choices=choices, fields=fields, empty_label=None)
 
             # e.g. ('field1', 'field2', ...)
             else:
@@ -394,14 +394,14 @@ def display_text(name, fltr):
                 # preference filter label over attribute name
                 choices = [(f, display_text(f, fltr)) for f, fltr in order_by]
                 fields = {fltr.name: f for f, fltr in order_by}
-                return OrderingFilter(choices=choices, fields=fields)
+                return OrderingFilter(choices=choices, fields=fields, empty_label=None)
 
         # opts.order_by = True
         order_by = filters.items()
 
         fields = [(fltr.name, f) for f, fltr in order_by]
         labels = {f: display_text(f, fltr) for f, fltr in order_by}
-        return OrderingFilter(fields=fields, field_labels=labels)
+        return OrderingFilter(fields=fields, field_labels=labels, empty_label=None)
 
     @classmethod
     def filters_for_model(cls, model, opts):


Test patch:
diff --git a/tests/test_conf.py b/tests/test_conf.py
--- a/tests/test_conf.py
+++ b/tests/test_conf.py
@@ -19,6 +19,15 @@ def test_help_text_filter(self):
     def test_help_text_exclude(self):
         self.assertTrue(settings.HELP_TEXT_EXCLUDE)
 
+    def test_empty_choice_label(self):
+        self.assertEqual(settings.EMPTY_CHOICE_LABEL, '---------')
+
+    def test_null_choice_label(self):
+        self.assertIsNone(settings.NULL_CHOICE_LABEL)
+
+    def test_null_choice_value(self):
+        self.assertEqual(settings.NULL_CHOICE_VALUE, 'null')
+
 
 class OverrideSettingsTests(TestCase):
 
diff --git a/tests/test_filtering.py b/tests/test_filtering.py
--- a/tests/test_filtering.py
+++ b/tests/test_filtering.py
@@ -125,12 +125,20 @@ class Meta:
 
 class ChoiceFilterTests(TestCase):
 
-    def test_filtering(self):
+    @classmethod
+    def setUpTestData(cls):
         User.objects.create(username='alex', status=1)
         User.objects.create(username='jacob', status=2)
         User.objects.create(username='aaron', status=2)
         User.objects.create(username='carl', status=0)
 
+        Article.objects.create(author_id=1, published=now())
+        Article.objects.create(author_id=2, published=now())
+        Article.objects.create(author_id=3, published=now())
+        Article.objects.create(author_id=4, published=now())
+        Article.objects.create(author_id=None, published=now())
+
+    def test_filtering(self):
         class F(FilterSet):
             class Meta:
                 model = User
@@ -156,11 +164,6 @@ def test_filtering_on_explicitly_defined_field(self):
 
         If you explicitly declare ChoiceFilter fields you **MUST** pass `choices`.
         """
-        User.objects.create(username='alex', status=1)
-        User.objects.create(username='jacob', status=2)
-        User.objects.create(username='aaron', status=2)
-        User.objects.create(username='carl', status=0)
-
         class F(FilterSet):
             status = ChoiceFilter(choices=STATUS_CHOICES)
 
@@ -182,6 +185,37 @@ class Meta:
         f = F({'status': '0'})
         self.assertQuerysetEqual(f.qs, ['carl'], lambda o: o.username, False)
 
+    def test_filtering_on_empty_choice(self):
+        class F(FilterSet):
+            class Meta:
+                model = User
+                fields = ['status']
+
+        f = F({'status': ''})
+        self.assertQuerysetEqual(f.qs,
+                                 ['aaron', 'alex', 'jacob', 'carl'],
+                                 lambda o: o.username, False)
+
+    def test_filtering_on_null_choice(self):
+        choices = [(u.pk, str(u)) for u in User.objects.order_by('id')]
+
+        class F(FilterSet):
+            author = ChoiceFilter(
+                choices=choices,
+                null_value='null',
+                null_label='NULL',
+            )
+
+            class Meta:
+                model = Article
+
+        # sanity check to make sure the filter is setup correctly
+        f = F({'author': '1'})
+        self.assertQuerysetEqual(f.qs, ['alex'], lambda o: str(o.author), False)
+
+        f = F({'author': 'null'})
+        self.assertQuerysetEqual(f.qs, [None], lambda o: o.author, False)
+
 
 class MultipleChoiceFilterTests(TestCase):
 
diff --git a/tests/test_filters.py b/tests/test_filters.py
--- a/tests/test_filters.py
+++ b/tests/test_filters.py
@@ -8,7 +8,7 @@
 import warnings
 
 from django import forms
-from django.test import TestCase
+from django.test import TestCase, override_settings
 
 from django_filters import filters, widgets
 from django_filters.fields import (
@@ -317,6 +317,82 @@ def test_default_field(self):
         field = f.field
         self.assertIsInstance(field, forms.ChoiceField)
 
+    def test_empty_choice(self):
+        # default value
+        f = ChoiceFilter(choices=[('a', 'a')])
+        self.assertEqual(f.field.choices, [
+            ('', '---------'),
+            ('a', 'a'),
+        ])
+
+        # set value, allow blank label
+        f = ChoiceFilter(choices=[('a', 'a')], empty_label='')
+        self.assertEqual(f.field.choices, [
+            ('', ''),
+            ('a', 'a'),
+        ])
+
+        # disable empty choice w/ None
+        f = ChoiceFilter(choices=[('a', 'a')], empty_label=None)
+        self.assertEqual(f.field.choices, [
+            ('a', 'a'),
+        ])
+
+    def test_null_choice(self):
+        # default is to be disabled
+        f = ChoiceFilter(choices=[('a', 'a')], )
+        self.assertEqual(f.field.choices, [
+            ('', '---------'),
+            ('a', 'a'),
+        ])
+
+        # set label, allow blank label
+        f = ChoiceFilter(choices=[('a', 'a')], null_label='')
+        self.assertEqual(f.field.choices, [
+            ('', '---------'),
+            ('null', ''),
+            ('a', 'a'),
+        ])
+
+        # set null value
+        f = ChoiceFilter(choices=[('a', 'a')], null_value='NULL', null_label='')
+        self.assertEqual(f.field.choices, [
+            ('', '---------'),
+            ('NULL', ''),
+            ('a', 'a'),
+        ])
+
+        # explicitly disable
+        f = ChoiceFilter(choices=[('a', 'a')], null_label=None)
+        self.assertEqual(f.field.choices, [
+            ('', '---------'),
+            ('a', 'a'),
+        ])
+
+    @override_settings(
+        FILTERS_EMPTY_CHOICE_LABEL='EMPTY LABEL',
+        FILTERS_NULL_CHOICE_LABEL='NULL LABEL',
+        FILTERS_NULL_CHOICE_VALUE='NULL VALUE', )
+    def test_settings_overrides(self):
+        f = ChoiceFilter(choices=[('a', 'a')], )
+        self.assertEqual(f.field.choices, [
+            ('', 'EMPTY LABEL'),
+            ('NULL VALUE', 'NULL LABEL'),
+            ('a', 'a'),
+        ])
+
+    def test_callable_choices(self):
+        def choices():
+            yield ('a', 'a')
+            yield ('b', 'b')
+
+        f = ChoiceFilter(choices=choices)
+        self.assertEqual(f.field.choices, [
+            ('', '---------'),
+            ('a', 'a'),
+            ('b', 'b'),
+        ])
+
 
 class MultipleChoiceFilterTests(TestCase):
 
@@ -1077,6 +1153,7 @@ def test_choices_unaltered(self):
         )
 
         self.assertSequenceEqual(f.field.choices, (
+            ('', '---------'),
             ('a', 'A'),
             ('b', 'B'),
         ))
@@ -1087,6 +1164,7 @@ def test_choices_from_fields(self):
         )
 
         self.assertSequenceEqual(f.field.choices, (
+            ('', '---------'),
             ('c', 'C'),
             ('-c', 'C (descending)'),
             ('d', 'D'),
@@ -1100,6 +1178,7 @@ def test_field_labels(self):
         )
 
         self.assertSequenceEqual(f.field.choices, (
+            ('', '---------'),
             ('c', 'foo'),
             ('-c', 'foo (descending)'),
             ('d', 'D'),
diff --git a/tests/test_filterset.py b/tests/test_filterset.py
--- a/tests/test_filterset.py
+++ b/tests/test_filterset.py
@@ -696,6 +696,9 @@ def test_method_self_is_parent(self):
         class F(FilterSet):
             f = CharFilter(method='filter_f')
 
+            class Meta:
+                model = User
+
             def filter_f(inner_self, qs, name, value):
                 self.assertIsInstance(inner_self, F)
                 self.assertIs(inner_self.request, request)
diff --git a/tests/test_forms.py b/tests/test_forms.py
--- a/tests/test_forms.py
+++ b/tests/test_forms.py
@@ -66,8 +66,10 @@ class Meta:
         f = F().form
         self.assertEqual(len(f.fields), 1)
         self.assertIn('status', f.fields)
-        self.assertEqual(sorted(f.fields['status'].choices),
-                         sorted(STATUS_CHOICES))
+        self.assertSequenceEqual(
+            f.fields['status'].choices,
+            (('', '---------'), ) + STATUS_CHOICES
+        )
 
     def test_form_fields_exclusion(self):
         class F(FilterSet):
@@ -106,7 +108,8 @@ class Meta:
     def test_form_fields_using_widget(self):
         class F(FilterSet):
             status = ChoiceFilter(widget=forms.RadioSelect,
-                                  choices=STATUS_CHOICES)
+                                  choices=STATUS_CHOICES,
+                                  empty_label=None)
 
             class Meta:
                 model = User
@@ -116,8 +119,10 @@ class Meta:
         self.assertEqual(len(f.fields), 2)
         self.assertIn('status', f.fields)
         self.assertIn('username', f.fields)
-        self.assertEqual(sorted(f.fields['status'].choices),
-                         sorted(STATUS_CHOICES))
+        self.assertSequenceEqual(
+            f.fields['status'].choices,
+            STATUS_CHOICES
+        )
         self.assertIsInstance(f.fields['status'].widget, forms.RadioSelect)
 
     def test_form_field_with_custom_label(self):