Repository: encode/django-rest-framework

Issue:
Support grouped choices.
**Update from @tomchristie**

**Closing off some related tickets and rolling them into this one...**
- [ ] Include core support.
- [ ] Render in browsable API. - #1636
- [ ] Display as metadata in response to `OPTIONS` requests. - #3101

---

One of our models has a Choicefield with following choices:

```
TYPE_CHOICES = (
        ('Default', (
            (1, 'Option 1'),
            (2, 'Option 2'),
            (3, 'Option 3'))),
        (4, 'Option 4'))
```

Now in the serializer if i have something _self.field.from_native(3)_, we get a ValidationError. Upon inspection of the DRF code we found that logic for valid_value method in the Choicefield is incorrect if you have nested sets with integer keys.

```
def valid_value(self, value):
        """
        Check to see if the provided value is a valid choice.
        """
        for k, v in self.choices:
            if isinstance(v, (list, tuple)):
                # This is an optgroup, so look inside the group for options
                for k2, v2 in v:
                    if value == smart_text(k2):
                        return True
            else:
                if value == smart_text(k) or value == k:
                    return True
        return False
```

Line _if value == smart_text(k2):_ should really be: _if value == smart_text(k2) or value == k2:_

Support grouped choices.
**Update from @tomchristie**

**Closing off some related tickets and rolling them into this one...**
- [ ] Include core support.
- [ ] Render in browsable API. - #1636
- [ ] Display as metadata in response to `OPTIONS` requests. - #3101

---

One of our models has a Choicefield with following choices:

```
TYPE_CHOICES = (
        ('Default', (
            (1, 'Option 1'),
            (2, 'Option 2'),
            (3, 'Option 3'))),
        (4, 'Option 4'))
```

Now in the serializer if i have something _self.field.from_native(3)_, we get a ValidationError. Upon inspection of the DRF code we found that logic for valid_value method in the Choicefield is incorrect if you have nested sets with integer keys.

```
def valid_value(self, value):
        """
        Check to see if the provided value is a valid choice.
        """
        for k, v in self.choices:
            if isinstance(v, (list, tuple)):
                # This is an optgroup, so look inside the group for options
                for k2, v2 in v:
                    if value == smart_text(k2):
                        return True
            else:
                if value == smart_text(k) or value == k:
                    return True
        return False
```

Line _if value == smart_text(k2):_ should really be: _if value == smart_text(k2) or value == k2:_



Discussion:
Yup, that seems valid to me.

DjangoCon sprint guidance - check out Django's docs on grouped choices. Demonstrate to yourself that the browsable API in REST framework does not currently support them. Make a pull request with the fix, including a screenshot of before and after. Ensure that `OPTIONS` requests continue to function identically before and after for endpoints that include choice fields.

I'll take a look at solving this at the DjangoCon Sprints today/tomorrow.

Ace.

Tom,

Thanks for the guidance on this. However I don't believe this issue affects the browsable API because integer values are always set as strings in the <option> tag. As a result of this, they are POSTed as strings, and appear as such in the request QueryDict. For this reason, the condition `value == smart_text(k2)` actually evaluates as `True` because smart_text() effectively typecasts `k2` as a string, matching the string sent by the browser and causing validation to succeed in this case.

However, using curl the issue is easily repeatable, e.g.: `curl -H "Content-Type: application/json" -d '{"this_is_an_integer_field": 1 }' http://127.0.0.1:8000/api/v1/widgets/` causes validation to fail unexpectedly as reported by @ameyc.

For this reason on the pull request I'll just include a targeted unit test that demonstrates the main failure case and succeeds with the change. Looking at the test suite structure I'm planning to add it to test_validation.py. Good for you?

Chris

Sounds okay.

Yup, that seems valid to me.

DjangoCon sprint guidance - check out Django's docs on grouped choices. Demonstrate to yourself that the browsable API in REST framework does not currently support them. Make a pull request with the fix, including a screenshot of before and after. Ensure that `OPTIONS` requests continue to function identically before and after for endpoints that include choice fields.

I'll take a look at solving this at the DjangoCon Sprints today/tomorrow.

Ace.

Tom,

Thanks for the guidance on this. However I don't believe this issue affects the browsable API because integer values are always set as strings in the <option> tag. As a result of this, they are POSTed as strings, and appear as such in the request QueryDict. For this reason, the condition `value == smart_text(k2)` actually evaluates as `True` because smart_text() effectively typecasts `k2` as a string, matching the string sent by the browser and causing validation to succeed in this case.

However, using curl the issue is easily repeatable, e.g.: `curl -H "Content-Type: application/json" -d '{"this_is_an_integer_field": 1 }' http://127.0.0.1:8000/api/v1/widgets/` causes validation to fail unexpectedly as reported by @ameyc.

For this reason on the pull request I'll just include a targeted unit test that demonstrates the main failure case and succeeds with the change. Looking at the test suite structure I'm planning to add it to test_validation.py. Good for you?

Chris

Sounds okay.


Resolving patch:
diff --git a/rest_framework/fields.py b/rest_framework/fields.py
--- a/rest_framework/fields.py
+++ b/rest_framework/fields.py
@@ -563,7 +563,7 @@ def valid_value(self, value):
             if isinstance(v, (list, tuple)):
                 # This is an optgroup, so look inside the group for options
                 for k2, v2 in v:
-                    if value == smart_text(k2):
+                    if value == smart_text(k2) or value == k2:
                         return True
             else:
                 if value == smart_text(k) or value == k:


Test patch:
diff --git a/tests/test_validation.py b/tests/test_validation.py
--- a/tests/test_validation.py
+++ b/tests/test_validation.py
@@ -1,5 +1,6 @@
 from __future__ import unicode_literals
 from django.core.validators import MaxValueValidator
+from django.core.exceptions import ValidationError
 from django.db import models
 from django.test import TestCase
 from rest_framework import generics, serializers, status
@@ -146,3 +147,42 @@ def test_max_value_validation_fail(self):
         response = view(request, pk=obj.pk).render()
         self.assertEqual(response.content, b'{"number_value": ["Ensure this value is less than or equal to 100."]}')
         self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
+
+
+class TestChoiceFieldChoicesValidate(TestCase):
+    CHOICES = [
+        (0, 'Small'),
+        (1, 'Medium'),
+        (2, 'Large'),
+    ]
+
+    CHOICES_NESTED = [
+        ('Category', (
+            (1, 'First'),
+            (2, 'Second'),
+            (3, 'Third'),
+        )),
+        (4, 'Fourth'),
+    ]
+
+    def test_choices(self):
+        """
+        Make sure a value for choices works as expected.
+        """
+        f = serializers.ChoiceField(choices=self.CHOICES)
+        value = self.CHOICES[0][0]
+        try:
+            f.validate(value)
+        except ValidationError:
+            self.fail("Value %s does not validate" % str(value))
+
+    def test_nested_choices(self):
+        """
+        Make sure a nested value for choices works as expected.
+        """
+        f = serializers.ChoiceField(choices=self.CHOICES_NESTED)
+        value = self.CHOICES_NESTED[0][1][0][0]
+        try:
+            f.validate(value)
+        except ValidationError:
+            self.fail("Value %s does not validate" % str(value))