Package: drf-dynamic-read
A utility to improve and optimise read operations(querying and serialization of data) for Django Rest Framework based applicationsOfficial version support:Django >=1.11Supported REST Framework versions >= 3.6.4Python >= 3.6CapabilitiesGives the ability to dynamically select required fields to be serializedWe can specify required fields to be serialized for a GET request via query paramsThis also reduces response size comparatively as we pick only necessary fieldsAbility to pick required fields through all kinds of nested relationships(many2one, many2many, reverse_lookups)Improves querying and reduces overall I/O load by a very good factorreduces overall number of queries required to serve a generic GET Request by arest_framework.viewsets.ModelViewSetPlug and PlaySimple API with minimal configurationsWhat it providesThis package provides following mixins:DynamicReadSerializerMixinprovides an API on top ofModelSerializerto provide required fields to be serialized(via kwargs)following kwargs can be passed to a model serializer inheriting this mixinfilter_fields: list of serializer field names which should be allowed for serializationomit_fields: list of serializer field names which should not be allowed for serializationDynamicReadSerializerMixin.optimize_queryset: a utility to return a optimized queryset by performing necessary select_related and prefetch_related based onfieldsandomit, below are the arguments to be passedfilter_fields: list of serializer field names which should be allowed for serializationomit_fields: list of serializer field names which should not be allowed for serializationqueryset: input queryset objectDynamicReadViewMixinprovides support on top ofrest_framework.viewsets.ModelViewSetto pick required fields to be serialized via query params of a GET request, these required fields are internally forwarded toDynamicReadSerializerMixinoptimize_queryset: static boolean attribute which decides whether to perform queryset optimization steps viaDynamicReadSerializerMixin.optimize_querysetfollowing query params can be passed for any GET request which is served by a model viewset inheriting this mixin:fields: serializer field names as comma seperated values which should be considered for serializationomit: serializer field names as comma seperated values which should not be considered for serializationInstallingpip install drf-dynamic-readUsageExample Entity Relationship:fromdjango.dbimportmodelsclassUser(models.Model)username=models.CharField()classEventType(models.Model)name=models.CharField()created_by=models.ForeignKey(User)classEventCause(models.Model)name=models.CharField()created_by=models.ForeignKey(User)classEvent(models.Model):type=models.ForeignKey(EventType)causes=models.ManyToManyField(EventCause)owner=models.OneToOneField(User)Example serializers for above ER:fromrest_frameworkimportserializersfromdynamic_read.serializersimportDynamicReadSerializerMixinclassUserSerializer(DynamicReadSerializerMixin,serializers.ModelSerializer):classMeta:model=models.Userfields="__all__"classEventTypeSerializer(DynamicReadSerializerMixin,serializers.ModelSerializer):created_by_id=serializers.PrimaryKeyRelatedField(queryset=EventType.objects.all(),write_only=True,source="created_by",)created_by=UserSerializer(read_only=True)classMeta:model=EventTypefields="__all__"classEventCauseSerializer(DynamicReadSerializerMixin,serializers.ModelSerializer):created_by_id=serializers.PrimaryKeyRelatedField(queryset=EventCause.objects.all(),write_only=True,source="created_by",)created_by=UserSerializer(read_only=True)classMeta:model=EventCausefields="__all__"classEventSerializer(DynamicReadSerializerMixin,serializers.ModelSerializer):type_id=serializers.PrimaryKeyRelatedField(queryset=EventType.objects.all(),write_only=True,source="type",)cause_ids=serializers.PrimaryKeyRelatedField(queryset=EventCause.objects.all(),write_only=True,source="cause",many=True)type=EventTypeSerializer(read_only=True)causes=EventCauseSerializer(read_only=True,many=True)created_by=UserSerializer(read_only=True)classMeta:model=Eventfields="__all__"Example views for above ER:fromdynamic_read.viewsimportDynamicReadBaseViewMixinfromrest_frameworkimportviewsetsfromrest_framework.routersimportDefaultRouterclassEventModelViewSet(viewsets.ModelViewSet,DynamicReadBaseViewMixin):queryset=Event.objects.all()serializer_class=EventSerializerrouter=DefaultRouter()router.register("/api/event_basic/",EventModelViewSet)A regular request returns all fields:GET /api/event_basic/Response:[{"id":1,"type":{"id":2,"name":"Type2","created_by":{"id":1,"username":"user1"}},"cause":[{"id":1,"name":"Cause1","created_by":{"id":1,"username":"user1"}},{"id":2,"name":"Cause2","created_by":{"id":2,"username":"user2"}}],"created_by":{"id":2,"username":"user2"}},]AGETrequest with thefieldsparameter returns only a subset of
the fields:GET/api/event_basic/?fields=id,typeResponse:[{"id":1,"type":{"id":2,"name":"Type2","created_by":{"id":1,"username":"user1"}}},{"id":2,"type":{"id":1,"name":"Type1","created_by":{"id":1,"username":"user1"}}}]fieldsparameter can spawn through the relationships also:GET/api/event_basic/?fields=id,type__name,cause__name,created_by__usernameResponse:[{"id":1,"type":{"name":"Type2"},"cause":[{"name":"Cause1"},{"name":"Cause2"}],"created_by":{"username":"user2"}},]AGETrequest with theomitparameter excludes specified fields(can also spawn through relationships just like the above example forfields).GET/api/event_basic/?omit=type,cause__created_by,created_by__idResponse:[{"id":1,"cause":[{"id":1,"name":"Cause1",},{"id":2,"name":"Cause2",}],"created_by":{"username":"user2"}},]All the above examples work in the same mechanism for detail routesQuery OptimizationNow first let’s consider this general request which returns all the fields:GET /api/event_basic/Total number of queries would be: 511 (Base query to return all the event objects)10 x 1 (fetch type for an event)10 x 1 (fetch created_by for an each type)10 x 1 (fetch all causes for an event)10 x 1 (fetch created_by for an event cause)10 x 1 (fetch owner for an event)Now let’s define a new view in views.py:fromdynamic_read.viewsimportDynamicReadViewMixinfromrest_frameworkimportviewsetsfromrest_framework.routersimportDefaultRouterclassEventModelViewSet(DynamicReadViewMixin,viewsets.ModelViewSet):queryset=Event.objects.all()serializer_class=EventSerializerclassEventOptimizedModelViewSet(DynamicReadViewMixin,viewsets.ModelViewSet)optimize_queryset=Truequeryset=Event.objects.all()serializer_class=EventSerializerrouter=DefaultRouter()router.register("/api/event_basic/",EventModelViewSet)router.register("/api/event_enhanced/",EventOptimizedModelViewSet)Now let’s try the optimized version:GET /api/event_enhanced/Total number of queries would be: 3.select_related("type","owner__created_by")1 (Query which gets all events inner joined with event types(inner joined with users), users).prefetch_related("causes__created_by")1 (Query to get all required event causes separately)1 (Query to get all users(created_by) for event causes)Now first let’s consider the above example withfields:GET/api/event_enhanced/?fields=type__name,owner__created_byTotal number of queries would be: 1.select_related("type","owner__created_by")1 (Query which gets all events inner joined with event types, users)TestingYet to write :)Planned featuresAPI aliasing, single view serving extended url patterns, each url pattern is an alias mapped to specific fields,omit valuesRestricting the scope of fields,omit w.r.t user defined permissions per APICreditsThis implementation is inspired fromdrf-dynamic-fieldsbydbrgnThanks to Rishab Jain for implementing caching in evaluation ofselect_related,prefetch_relatedfor aQuerySetw.r.t fields, omitThanks to Martin Garrix for his amazing music, sourcing all the necessary dopamineLicenseMIT license, seeLICENSEfile.