Repository: comic/grand-challenge.org

Issue:
Create a simple interface for fetching datatypes schemas on grand-challenge 
**Problem**
The grand-challenge datatypes are currently only stored in the [gc-api](https://github.com/DIAGNijmegen/rse-gcapi/tree/master/gcapi/schemas) repository. However, the information is required by other libraries as well. Duplication of this information seems bad.

**Solution**
* [x] It would be nice to have this information in a central location like grand-challenge and provide a simple GET interface to allow the libraries /comic/evalutils and /DIAGNijmegen/rse-gcapi/ to fetch and cache this information (only a few kbs) from grand-challenge.
* [x] The answer type schemas should be added to the generated schema



Discussion:
From #1048

> Note that this PR does not close #1017, as the answer type schemas are not added to the generated schema so needs to be extended, see https://drf-yasg.readthedocs.io/en/stable/custom_spec.html
Another problem with this - the schema is empty for unauthorised users. You need to add `public=True` to `get_schema_view`.
@jmsmkn Today I had a thorough look at adding the answer type schemas together with Harm to the generated schemas. We figured that the choice was made to support two fields: answer_type (4 chars) and answer (JSON serializable string). If this design is retained, it complicates the automatic generation of schemas somewhat and we are not entirely sure how to proceed since there seem to be many options.

Looking at the custom specifications documentation we can use three approaches:
* Write a custom OpenAPISchemaGenerator subclass to `/api/urls.py` which adds an AnswerType schema to the definitions, without additional serializers. (This could be either the whole AnswerType schema, its subdefinitions or both). This solution seems the simplest, but it might be somewhat messy to inject the schemas.
* We could subclass `JSONField` for `reader_studies/models.py::Answer.answer` and call it something like `AnswerTypeField` and define the swagger schemas directly (should be picked up by the auto-generation) ([example](https://github.com/axnsan12/drf-yasg/issues/396))
```python 
class AnswerTypeField(JSONField):
    class Meta:
        swagger_schema_fields = {

        }

```
This approach allows automatic detection by the auto-generation but might require designing multiple serializers for the individual AnswerTypes. It is also uncertain if this approach works, or would introduce duplicated information.
* Make all the answer types explicit models, which would allow automatic detection. This would require a lot of refactoring, migration and appears to go against the initial design decision.

ps. as a side note, the enum field for `answer_type` on the `Question` model class gets overwritten by the serializer, removing the enum from the generated schemas. I guess this was not intended?
> as a side note, the enum field for answer_type on the Question model class gets overwritten by the serializer, removing the enum from the generated schemas. I guess this was not intended?

Not too sure what you mean here. In serializers.py:

```
class QuestionSerializer(HyperlinkedModelSerializer):
    answer_type = CharField(source="get_answer_type_display")
    ...
    
    class Meta:
        ...
        fields = (
            "answer_type",
            ...
        )
```

This means that we will use `Question.get_answer_type_display()` for the value of `answer_type` (this is an implicit function on the model generated by the Django choice field), meaning that the values of the serialized field are in `{"Single line text",
        "Multi line text",
        "Bool",
        "Heading",
        "2D bounding box",
        "Distance measurement",
 "Multiple distance measurements"}
        ` rather than in `{"STXT",
    "MTXT",
    "BOOL",
    "HEAD",
    "2DBB",
    "DIST",
    "MDIS"}` which are less meaningful to the outside world (these would be the values that the frontend would need to work with if we used `Question.answer_type` directly). 
> We could subclass JSONField for reader_studies/models.py::Answer.answer and call it something like AnswerTypeField and define the swagger schemas directly (should be picked up by the auto-generation) (example)

My guess is that this is what we want, and that we can more or less drop `ANSWER_TYPE_ANNOTATIONS_SCHEMA` directly in there.
@jmsmkn The generated schema for `Question.answer_type` now look like this:
```
      "Question":{
         "required":[
            "answer_type",
            "form_direction",
            "image_port",
            "question_text"
         ],
         "type":"object",
         "properties":{
            "answer_type":{
               "title":"Answer type",
               "type":"string",
               "minLength":1
            },
```

whereas for normal enumerated fields this looks like, e.g.:

```
            "color_interpolation":{
               "title":"Color interpolation",
               "type":"string",
               "enum":[
                  "RGB",
                  "HLS",
                  "HLSpos",
                  "HLSneg",
                  "Constant"
               ]
```

It appears that the definition in the model (below) gets overwritten by the definition in the Serializer and removes the enumeration from showing up.

```
    answer_type = models.CharField(
        max_length=4,
        choices=ANSWER_TYPE_CHOICES,
        default=ANSWER_TYPE_SINGLE_LINE_TEXT,
    )
```

Maybe this can be easily added by setting the choices attribute for the serializer?

> Maybe this can be easily added by setting the choices attribute for the serializer?

Have a go at adding it, the values will be `{c[1] for c in ANSWER_TYPE_CHOICES}` rather than `{c[0] for c in ANSWER_TYPE_CHOICES}`.

> > We could subclass JSONField for reader_studies/models.py::Answer.answer and call it something like AnswerTypeField and define the swagger schemas directly (should be picked up by the auto-generation) (example)
> 
> My guess is that this is what we want, and that we can more or less drop `ANSWER_TYPE_ANNOTATIONS_SCHEMA` directly in there.

Yes, ideally I would like to replace the structure and replace it with valid `Schema` (to avoid definition duplication). This would involve adding the other subtypes (bool, string, multi-string, etc...) for type checking. This could potentially also clean up the code quite a bit. Let's hope that `drf-yasg` plays nice with the nested definitions.


Also, we use the same construct elsewhere, so if it works for the answer type also update `form_direction`, `image_port`, `default_orientation` (workstation configs) and `status` (workstations).
@jmsmkn I failed to notice that we were dealing with two different CharField classes. It appears that overriding the attribute in the Serializer with a `rest_framework.CharField` is differently interpreted than the `models.CharField`  by `drf-yasg`. The former has no attribute `choices` and `drf-yasg` will not pick up the enum if it is used. Dropping the CharField assignment in the Serializer does seem to work. However, I didn't test if that will break other stuff.
It would break CIRRUS and any other API consumers..! We did it this way so as not to expose the internal database values for the choice field and give the front end some human-meaningful values to work with.
@jmsmkn Alright, we can't do it that way then.
I did find an alternative workaround for this issue, it is a bit involved though. 

* We can specify swagger_schema_fields in the serializer instead. Normally, this would override the entire properties dictionary instead of updating it, but using [this 'hack'(https://github.com/axnsan12/drf-yasg/issues/291#issuecomment-488574186)] we can fix this behavior.

No setting the swagger_schema_fileds in the serializer:

```
        swagger_schema_fields = {
            "properties": {
                "answer_type": Schema(**{
                    "enum": [c[1] for c in Question.ANSWER_TYPE_CHOICES],
                    "title": "Answer type",
                    "type": "string",
                    "minLength": 1
                })
            }
        }

```


This will yield:

```
Schema([('required',
         ['answer_type', 'form_direction', 'image_port', 'question_text']),
        ('type', 'object'),
        ('properties',
         {'answer_type': Schema([('title', 'Answer type'),
                                 ('type', 'string'),
                                 ('enum',
                                  ['Single line text',
                                   'Multi line text',
                                   'Bool',
                                   'Heading',
                                   '2D bounding box',
                                   'Distance measurement',
                                   'Multiple distance measurements']),
                                  ('minLength', 1)]),

```

JSON formatted:
```
{'properties': {'answer_type': {'enum': ['Single line text',
                                         'Multi line text',
                                         'Bool',
                                         'Heading',
                                         '2D bounding box',
                                         'Distance measurement',
                                         'Multiple distance measurements'],
                                'minLength': 1,
                                'title': 'Answer type',
                                'type': 'string'},
                'api_url': {'readOnly': True,
                            'title': 'Api url',
                            'type': 'string'},
                'form_direction': {'minLength': 1,
                                   'title': 'Form direction',
                                   'type': 'string'},

```

The tests seem to indicate that these changes are fine. We should be able to apply this construct elsewhere as suggested. I am not a big fan of injecting the hack, but it makes the `swagger_schemas_fields` much more useful in my opinion.



LGTM

On Tue, 7 Jan 2020 at 11:08, Sil van de Leemput <notifications@github.com>
wrote:

> @jmsmkn <https://github.com/jmsmkn> Alright, we can't do it that way then.
> I did find an alternative workaround for this issue, it is a bit involved
> though.
>
>    - We can specify swagger_schema_fields in the serializer instead.
>    Normally, this would override the entire properties dictionary instead of
>    updating it, but using [this 'hack'(https://github.com/
>    axnsan12/drf-yasg/issues/291
>    <https://github.com/axnsan12/drf-yasg/issues/291>#issuecomment-488574186)]
>    we can fix this behavior.
>
> No setting the swagger_schema_fileds in the serializer:
>
>         swagger_schema_fields = {
>             "properties": {
>                 "answer_type": Schema(**{
>                     "enum": [c[1] for c in Question.ANSWER_TYPE_CHOICES],
>                     "title": "Answer type",
>                     "type": "string",
>                     "minLength": 1
>                 })
>             }
>         }
>
>
> This will yield:
>
> Schema([('required',
>          ['answer_type', 'form_direction', 'image_port', 'question_text']),
>         ('type', 'object'),
>         ('properties',
>          {'answer_type': Schema([('title', 'Answer type'),
>                                  ('type', 'string'),
>                                  ('enum',
>                                   ['Single line text',
>                                    'Multi line text',
>                                    'Bool',
>                                    'Heading',
>                                    '2D bounding box',
>                                    'Distance measurement',
>                                    'Multiple distance measurements']),
>                                   ('minLength', 1)]),
>
>
> JSON formatted:
>
> {'properties': {'answer_type': {'enum': ['Single line text',
>                                          'Multi line text',
>                                          'Bool',
>                                          'Heading',
>                                          '2D bounding box',
>                                          'Distance measurement',
>                                          'Multiple distance measurements'],
>                                 'minLength': 1,
>                                 'title': 'Answer type',
>                                 'type': 'string'},
>                 'api_url': {'readOnly': True,
>                             'title': 'Api url',
>                             'type': 'string'},
>                 'form_direction': {'minLength': 1,
>                                    'title': 'Form direction',
>                                    'type': 'string'},
>
>
> The tests seem to indicate that these changes are fine. We should be able
> to apply this construct elsewhere as suggested. I am not a big fan of
> injecting the hack, but it makes the swagger_schemas_fields much more
> useful in my opinion.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/comic/grand-challenge.org/issues/1017?email_source=notifications&email_token=ADATGM3MIDEOQEQ7VH33FI3Q4RICBA5CNFSM4JUVNUF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIILTUI#issuecomment-571521489>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ADATGM2KDMZYRKA4RN6NAZDQ4RICBANCNFSM4JUVNUFQ>
> .
>


Resolving patch:
diff --git a/app/grandchallenge/reader_studies/serializers.py b/app/grandchallenge/reader_studies/serializers.py
--- a/app/grandchallenge/reader_studies/serializers.py
+++ b/app/grandchallenge/reader_studies/serializers.py
@@ -8,7 +8,12 @@
 
 from grandchallenge.api.swagger import swagger_schema_fields_for_charfield
 from grandchallenge.cases.models import Image
-from grandchallenge.reader_studies.models import Answer, Question, ReaderStudy
+from grandchallenge.reader_studies.models import (
+    ANSWER_TYPE_SCHEMA,
+    Answer,
+    Question,
+    ReaderStudy,
+)
 
 
 class QuestionSerializer(HyperlinkedModelSerializer):
@@ -123,3 +128,6 @@ class Meta:
             "pk",
             "question",
         )
+        swagger_schema_fields = {
+            "properties": {"answer": {"title": "Answer", **ANSWER_TYPE_SCHEMA}}
+        }


Test patch:
diff --git a/app/tests/api_tests/test_urls.py b/app/tests/api_tests/test_urls.py
--- a/app/tests/api_tests/test_urls.py
+++ b/app/tests/api_tests/test_urls.py
@@ -8,6 +8,7 @@
 from rest_framework.serializers import ModelSerializer
 
 import grandchallenge.api.urls
+from grandchallenge.reader_studies.models import ANSWER_TYPE_SCHEMA
 from grandchallenge.reader_studies.serializers import QuestionSerializer
 from grandchallenge.subdomains.utils import reverse
 from tests.utils import assert_viewname_status
@@ -29,11 +30,17 @@ def test_api_docs_generation(client, schema, schema_format):
     )
     if schema_format is not None:
         assert len(response.data["paths"]) > 0
+        check_answer_type_schema_from_response(response)
         check_response_schema_formatting(response)
     else:
         assert len(response.content) > 0
 
 
+def check_answer_type_schema_from_response(response):
+    schema = response.data["definitions"]["Answer"]["properties"]["answer"]
+    assert {"title": "Answer", **ANSWER_TYPE_SCHEMA} == schema
+
+
 def check_response_schema_formatting(response):
     definitions = response.data["definitions"]
     serializers = extract_all_api_exposed_model_serializers()