Package: drf-serializer-prefetch
drf-serializer-prefetchAn automatic prefetcher that looks at the serializer fields and determines what needs to be prefetched accordingly.InstallationTo install, callpip install drf-serializer-prefetch.UsageIn its most simple form, the serializer prefetch can be used by simply addingPrefetchingSerializerMixinto the class definition of the model serializer you want to allow automatic prefetching for like so:fromrest_frameworkimportserializersfromserializer_prefetchimportPrefetchingSerializerMixinclassSomeSerializer(PrefetchingSerializerMixin,serializer.ModelSerializer):classMeta:model=SomeModelfields=('some','fields','are','defined',)Using it like this, the Prefetching Serializer will be able to see what fields will be returned and prefetch them accordingly.However, SerializerMethodFields are an issue for the Prefetching Serializer, as those can make calls that are only done at run time. The Prefetching Serializer does not make any kind of file analysis, and as such cannot know what will happen in those methods. If you are calling a related model in that function, you can define eitherselect_relatedorprefetch_relatedon the model. The fields defined here will also be taken into account. As for usual with Django, you can go as deep as you need here. For example, you could adduser, or you could adduser__pizza_set, if you were to loop over the user's pizzas.For example:fromrest_frameworkimportserializersfromserializer_prefetchimportPrefetchingSerializerMixinclassSomeSerializer(PrefetchingSerializerMixin,serializer.ModelSerializer):select_related=('other_model',)other_field=serializers.SerializerMethodField()defget_other_field(self,obj):returnobj.other_model.other_fieldclassMeta:model=SomeModelfields=('some','fields','are','defined',)If you need to add some logic to when the prefetching should happen, you can also defineget_select_relatedorget_prefetch_related:fromrest_frameworkimportserializersfromserializer_prefetchimportPrefetchingSerializerMixinclassSomeSerializer(PrefetchingSerializerMixin,serializer.ModelSerializer):defget_select_related(self):return('other_model',)other_field=serializers.SerializerMethodField()defget_other_field(self,obj):returnobj.other_model.other_fieldclassMeta:model=SomeModelfields=('some','fields','are','defined',)Note that because this is run before the queryset has been split into individual models, we cannot use object logic here. If you need a field for some models only, you can either fetch it for all of them, or fetch it for none of them.Sometimes, other serializers can be called in aSerializerMethodField. For this, you can either defineadditional_serializerslike so:fromrest_frameworkimportserializersfromserializer_prefetchimportPrefetchingSerializerMixinclassSomeSerializer(PrefetchingSerializerMixin,serializer.ModelSerializer):additional_serializers=({'relation_and_field':'other_model','serializer':OtherModelSerializer(),},)other_field=serializers.SerializerMethodField()defget_other_field(self,obj):returnOtherModelSerializer(obj.other_model,auto_prefetch=False)classMeta:model=SomeModelfields=('some','fields','are','defined',)Notice the partauto_prefetch=Falsein theOtherModelSerializercall. This is because drf-auto-prefetch uses parents to know if the serializer is at the higher-most level, so that the fetching is only done once. In this case however, the serializer has no parent, but the prefetching is already done by SomeSerializer, so we do not want to do it again. We can tell it to not prefetch by settingauto_prefetchtoFalse.As forselect_relatedandprefetch_related, you can defineget_additional_serializersinstead:fromrest_frameworkimportserializersfromserializer_prefetchimportPrefetchingSerializerMixinclassSomeSerializer(PrefetchingSerializerMixin,serializer.ModelSerializer):defget_additional_serializers:return({'relation_and_field':'other_model','serializer':OtherModelSerializer(),},)other_field=serializers.SerializerMethodField()defget_other_field(self,obj):returnOtherModelSerializer(obj.other_model,auto_prefetch=False)classMeta:model=SomeModelfields=('some','fields','are','defined',)This can be useful to avoid circular dependencies or to add some logic to the prefetching.As of version 1.1.0, the prefetching serializer now allows passing aPrefetchobject to prefetch_related and to therelation_and_fieldvalue ofadditional_serializers. This is very useful to avoid fetching more than needed, espescially when filtering is needed in aSerializerMethodField. Using this new feature is as simple as using the string lookup:fromrest_frameworkimportserializersfromserializer_prefetchimportPrefetchingSerializerMixinclassSomeSerializer(PrefetchingSerializerMixin,serializer.ModelSerializer):prefetch_related=(Prefetch('other_field',queryset=OtherField.objects.filter(some_data='some_value'),to_attr='other_field_filtered'),)other_field_filtered=serializers.ListField(child=serializers.CharField())classMeta:model=SomeModelfields=('some','fields','are','defined',)This can also be used to filter out the data before sending it to another serializer:fromrest_frameworkimportserializersfromserializer_prefetchimportPrefetchingSerializerMixinclassSomeSerializer(PrefetchingSerializerMixin,serializer.ModelSerializer):prefetch_related=(Prefetch('other_field',queryset=OtherField.objects.filter(some_data='some_value'),to_attr='other_field_filtered'),)other_field_filtered=OtherFieldSerializer(many=True)classMeta:model=SomeModelfields=('some','fields','are','defined',)IMPORTANT NOTE: Because of an issue with Django, behaviour could be inconsistant when using to_attr with Prefetch objects. For more information, see:https://code.djangoproject.com/ticket/34791As of version 1.1.3, it is now possible to set a new parameter calledforce_prefetchon the serializer. This parameter allows forcing a parameter that would otherwise be in the select_related to be in the prefetch_related. This can be useful when the join time would be longer than just fetching those objects, for instance in the case where there are only 2 or 3 objects to be fetched in total. In certain situations, it is faster to prefetch than to select, so this allows more control over the method used.As of version 1.1.6, you can now passprefetch_sourcedirectly to the serializer to tell it to prefetch from another source than the one that is used to get the data. This is especially useful if your source is a property or a callable. By default, the serializer prefetch cannot know what field is actually getting fetched, and will ignore this prefetch entirely as a result.prefetch_sourceallows to let it know which model field should be prefetched.If theprefetch_sourcepassed is not a valid model field, aValueErrorwill be raised.Special casesThere are a few situations where you might want to be able to customize the behaviour more. Here are some of the ways you can tweak the Prefetching Serializer to fit the needs of your project.Adding logic to the PrefetchingListSerializerIf you had a ListSerializer for you Serializer, you will most likely want to keep its logic. This can be done by simply inheriting fromPrefetchingListSerializerin that ListSerializer rather than inheriting fromserializers.ListSerializer. Note that if you do not do this, but add thePrefetchingSerializerMixinto the main Serializer, you will get an error saying that the list_serializer_class must inherit fromPrefetchingListSerializerto be used with thePrefetchingSerializerMixin. This is because the behaviour of the prefetching depends on it.Adding compatibility with django-zen-queries or other librariesThe goal of this library is to make it easy to not have to think about prefetching. However, this comes with the potential danger of not thinking enough about prefetching. To avoid discovering issues only once in production, you can add django-zen-queries in the mix. To do so, simply extend thePrefetchingListSerializerandPrefetchingSerializerMixinby defining a new mixin that will be inherited by both. In this mixin, you can override thequeryset_after_prefetchandcall_to_representationmethods to add some behaviour right after the prefetching has been done on the queryset and right before or after callingsuper().to_representation(instance)on the serializer respectively. For django-zen-queries, it looks something like this:fromcontextlibimportnullcontextfromdjango.confimportsettingsimportserializer_prefetchfromzen_queriesimportfetch,queries_disabledclassZenQueriesPrefetchingMixin:defqueryset_after_prefetch(self,queryset):fetch(queryset)returnquerysetdefcall_to_representation(self,instance):withqueries_disabled()ifsettings.DEBUGelsenullcontext():returnsuper().to_representation(instance)classPrefetchingListSerializer(ZenQueriesPrefetchingMixin,serializer_prefetch.PrefetchingListSerializer):passclassPrefetchingSerializerMixin(ZenQueriesPrefetchingMixin,serializer_prefetch.PrefetchingSerializerMixin):default_list_serializer_class=PrefetchingListSerializerThe same logic can be applied to other libraries that could need to interact with the queryset between the time it is prefetched and the time it is split into models. Note thatqueryset_after_prefetchis only ever called if the instance passed to the ListSerializer is a Queryset, so you do not need to worry about checking for the type. Similarly,call_to_representationis only called if some prefetching was done. If none was done, either because it was not the furthermost parent or because auto_prefetch has been set toFalse, then this method will not be called. If you need a method to always be called, you can define ato_representationmethod in the newPrefetchingListSerializeryou defined. Do not forget to call the super method!Important noteWhile the Prefetching Serializer does work on regular django-rest-framework serializers, it is really intended to be used with theirModelSerializer. If you do use it with a regular Serializer, you will need to defineselect_relatedandprefetch_relatedfor all the fields used. Note that this is still better than having the logic inget_querysetin the viewset, as it is kept with its own serializer and will be used properly if the serializer is nested.