Package: databackend
databackendThedatabackendpackage allows you to register a subclass, without
needing to import the subclass itself. This is useful for implementing
actions over optional dependencies.ExampleFor this example, we’ll implement a function,fill_na(), that fills in
missing values in a DataFrame. It works with DataFrame objects from two
popular libraries:pandasandpolars. Importantly, neither library
needs to be installed.SetupThe code below defines “abstract” parent classes for each of the
DataFrame classes in the two libraries.fromdatabackendimportAbstractBackendclassAbstractPandasFrame(AbstractBackend):_backends=[("pandas","DataFrame")]classAbstractPolarsFrame(AbstractBackend):_backends=[("polars","DataFrame")]Note that the abstract classes can be used as stand-ins for the real
thing inissubclass()andisinstance.frompandasimportDataFrameissubclass(DataFrame,AbstractPandasFrame)isinstance(DataFrame(),AbstractPandasFrame)TrueSimple fill_na: isinstance to switch behaviorThefill_na()function below uses custom handling for pandas and
polars.deffill_na(data,x):ifisinstance(data,AbstractPolarsFrame):returndata.fill_nan(x)elifisinstance(data,AbstractPandasFrame):returndata.fillna(x)else:raiseNotImplementedError()Notice that neitherpandasnorpolarsneed to be imported when
definingfill_na().Here is an example of callingfill_na()on both kinds of DataFrames.# test polars ----importpolarsaspldf=pl.DataFrame({"x":[1,2,None]})fill_na(df,3)# test pandas ----importpandasaspddf=pd.DataFrame({"x":[1,2,None]})fill_na(df,3)x
0  1.0
1  2.0
2  3.0The key here is that a user could have only pandas, or only polars,
installed. Importantly, doing the isinstance checks do not import any
libraries!Advanced fill_na: generic function dispatchdatabackendshines when combined withgeneric function
dispatch.
This is a programming approach where you declare a function
(e.g.fill_na()), and then register each backend specific
implementation on the function.Python has a built-in function implementing this calledfunctools.singledispatch.Here is an example of the previousfill_na()function written using
it.fromfunctoolsimportsingledispatch@singledispatchdeffill_na2(data,x):raiseNotImplementedError(f"No support for class:{type(data)}")# handle polars ----@fill_na2.registerdef_(data:AbstractPolarsFrame,x):returndata.fill_nan(x)# handle pandas ----@fill_na2.registerdef_(data:AbstractPandasFrame,x):returndata.fillna(x)Note two important decorators:@singledispatchdefines a default function. This gets called if no
specific implementations are found.@fill_na2.registerdefines specific versions of the function.Here’s an example of it in action.# example ----importpandasaspdimportpolarsaspldf=pl.DataFrame({"x":[1,2,None]})fill_na2(df,3)df=pd.DataFrame({"x":[1,2,None]})fill_na2(df,3)x
0  1.0
1  2.0
2  3.0How it worksUnder the hood,AbstractBackendbehaves similarly to python’s builtinabc.ABCclass.fromabcimportABCclassMyABC(ABC):passfromioimportStringIOMyABC.register(StringIO)# StringIO is a "virtual subclass" of MyABCisinstance(StringIO("abc"),MyABC)TrueThe key difference is that you can specify the virtual subclass using
the tuple("<mod_name>", "<class_name>").Whenissubclass(SomeClass, AbstractBackend)runs, then…The standard ABC caching mechanism is checked, and potentially
returns the answer immediately.Otherwise, a subclass hook cycles through registered backends.The hook runs the subclass check for any backends that are imported
(e.g. are insys.modules).Technically,AbstractBackendinherits all the useful metaclass things
fromabc.ABCMeta, so these can be used also.