Package: catalogue
catalogue: Super lightweight function registries for your librarycatalogueis a tiny, zero-dependencies library that makes it easy toadd
function (or object) registriesto your code. Function registries are helpful
when you have objects that need to be both easily serializable and fully
customizable. Instead of passing a function into your object, you pass in an
identifier name, which the object can use to lookup the function from the
registry. This makes the object easy to serialize, because the name is a simple
string. If you instead saved the function, you'd have to use Pickle for
serialization, which has many drawbacks.⏳ Installationpipinstallcataloguecondainstall-cconda-forgecatalogue⚠️Important note:cataloguev2.0+ is only compatible with Python 3.6+.
For Python 2.7+ compatibility, usecataloguev1.x.👩‍💻 UsageLet's imagine you're developing a Python package that needs to load data
somewhere. You've already implemented some loader functions for the most common
data types, but you want to allow the user to easily add their own. Usingcatalogue.createyou can create a new registry under the namespaceyour_package→loaders.# YOUR PACKAGEimportcatalogueloaders=catalogue.create("your_package","loaders")This gives you aloaders.registerdecorator that your users can import and
decorate their custom loader functions with.# USER CODEfromyour_packageimportloaders@loaders.register("custom_loader")defcustom_loader(data):# Load something here...returndataThe decorated function will be registered automatically and in your package,
you'll be able to access all loaders by callingloaders.get_all.# YOUR PACKAGEdefload_data(data,loader_id):print("All loaders:",loaders.get_all())# {"custom_loader": <custom_loader>}loader=loaders.get(loader_id)returnloader(data)The user can now refer to their custom loader using only its string name
("custom_loader") and your application will know what to do and will use their
custom function.# USER CODEfromyour_packageimportload_dataload_data(data,loader_id="custom_loader")❓ FAQBut can't the user just pass in thecustom_loaderfunction directly?Sure, that's the more classic callback approach. Instead of a string ID,load_datacould also take a function, in which case you wouldn't need a
package like this.cataloguehelps you when you need to produce a serializable
record of which functions were passed in. For instance, you might want to write
a log message, or save a config to load back your object later. Withcatalogue, your functions can be parameterized by strings, so logging and
serialization remains easy – while still giving you full extensibility.How do I make sure all of the registration decorators have run?Decorators normally run when modules are imported. Relying on this side-effect
can sometimes lead to confusion, especially if there's no other reason the
module would be imported. One solution is to useentry points.For instance, inspaCywe're starting to use function
registries to make the pipeline components much more customizable. Let's say one
user, Jo, develops a better tagging model using new machine learning research.
End-users of Jo's package should be able to writespacy.load("jo_tagging_model"). They shouldn't need to remember to writeimport jos_tagged_modelfirst, just to run the function registries as a
side-effect. With entry points, the registration happens at install time – so
you don't need to rely on the import side-effects.🎛 APIfunctioncatalogue.createCreate a new registry for a given namespace. Returns a setter function that can
be used as a decorator or called with a name andfunckeyword argument. Ifentry_points=Trueis set, the registry will check forPython entry pointsadvertised for the given namespace, e.g. the entry point groupspacy_architecturesfor the namespace"spacy", "architectures", inRegistry.getandRegistry.get_all. This allows other packages to
auto-register functions.ArgumentTypeDescription*namespacestrThe namespace, e.g."spacy"or"spacy", "architectures".entry_pointsboolWhether to check for entry points of the given namespace and pre-populate the global registry.RETURNSRegistryTheRegistryobject with methods to register and retrieve functions.architectures=catalogue.create("spacy","architectures")# Use as decorator@architectures.register("custom_architecture")defcustom_architecture():pass# Use as regular functionarchitectures.register("custom_architecture",func=custom_architecture)classRegistryThe registry object that can be used to register and retrieve functions. It's
usually created internally when you callcatalogue.create.methodRegistry.__init__Initialize a new registry. Ifentry_points=Trueis set, the registry will
check forPython entry pointsadvertised for the given namespace, e.g. the entry point groupspacy_architecturesfor the namespace"spacy", "architectures", inRegistry.getandRegistry.get_all.ArgumentTypeDescriptionnamespaceTuple[str]The namespace, e.g."spacy"or"spacy", "architectures".entry_pointsboolWhether to check for entry points of the given namespace ingetandget_all.RETURNSRegistryThe newly created object.# User-facing APIarchitectures=catalogue.create("spacy","architectures")# Internal APIarchitectures=Registry(("spacy","architectures"))methodRegistry.__contains__Check whether a name is in the registry.ArgumentTypeDescriptionnamestrThe name to check.RETURNSboolWhether the name is in the registry.architectures=catalogue.create("spacy","architectures")@architectures.register("custom_architecture")defcustom_architecture():passassert"custom_architecture"inarchitecturesmethodRegistry.__call__Register a function in the registry's namespace. Can be used as a decorator or
called as a function with thefunckeyword argument supplying the function to
register. Delegates toRegistry.register.methodRegistry.registerRegister a function in the registry's namespace. Can be used as a decorator or
called as a function with thefunckeyword argument supplying the function to
register.ArgumentTypeDescriptionnamestrThe name to register under the namespace.funcAnyOptional function to register (if not used as decorator).RETURNSCallableThe decorator that takes one argument, the name.architectures=catalogue.create("spacy","architectures")# Use as decorator@architectures.register("custom_architecture")defcustom_architecture():pass# Use as regular functionarchitectures.register("custom_architecture",func=custom_architecture)methodRegistry.getGet a function registered in the namespace.ArgumentTypeDescriptionnamestrThe name.RETURNSAnyThe registered function.custom_architecture=architectures.get("custom_architecture")methodRegistry.get_allGet all functions in the registry's namespace.ArgumentTypeDescriptionRETURNSDict[str, Any]The registered functions, keyed by name.all_architectures=architectures.get_all()# {"custom_architecture": <custom_architecture>}methodRegistry.get_entry_pointsGet registered entry points from other packages for this namespace. The name of
the entry point group is the namespace joined by_.ArgumentTypeDescriptionRETURNSDict[str, Any]The loaded entry points, keyed by name.architectures=catalogue.create("spacy","architectures",entry_points=True)# Will get all entry points of the group "spacy_architectures"all_entry_points=architectures.get_entry_points()methodRegistry.get_entry_pointCheck if registered entry point is available for a given name in the namespace
and load it. Otherwise, return the default value.ArgumentTypeDescriptionnamestrName of entry point to load.defaultAnyThe default value to return. Defaults toNone.RETURNSAnyThe loaded entry point or the default value.architectures=catalogue.create("spacy","architectures",entry_points=True)# Will get entry point "custom_architecture" of the group "spacy_architectures"custom_architecture=architectures.get_entry_point("custom_architecture")methodRegistry.findFind the information about a registered function, including the module and path
to the file it's defined in, the line number and the docstring, if available.ArgumentTypeDescriptionnamestrName of the registered function.RETURNSDict[str, Union[str, int]]The information about the function.importcataloguearchitectures=catalogue.create("spacy","architectures",entry_points=True)@architectures("my_architecture")defmy_architecture():"""This is an architecture"""passinfo=architectures.find("my_architecture")# {'module': 'your_package.architectures',#  'file': '/path/to/your_package/architectures.py',#  'line_no': 5,#  'docstring': 'This is an architecture'}functioncatalogue.check_existsCheck if a namespace exists.ArgumentTypeDescription*namespacestrThe namespace, e.g."spacy"or"spacy", "architectures".RETURNSboolWhether the namespace exists.