Package: plux
Pluxplux is the dynamic code loading framework used inLocalStack.OverviewPlux builds a higher-level plugin mechanism aroundPython's entry point mechanism.
It provides tools to load plugins from entry points at run time, and to discover entry points from plugins at build time (so you don't have to declare entry points statically in yoursetup.py).Core conceptsPluginSpec: describes aPlugin. Each plugin has a namespace, a unique name in that namespace, and aPluginFactory(something that createsPluginthe spec is describing.
In the simplest case, that can just be the Plugin's class).Plugin: an object that exposes ashould_loadandloadmethod.
Note that it does not function as a domain object (it does not hold the plugins lifecycle state, like initialized, loaded, etc..., or other metadata of the Plugin)PluginFinder: finds plugins, either at build time (by scanning the modules usingpkgutilandsetuptools) or at run time (reading entrypoints of the distribution usingstevedore)PluginManager: manages the run time lifecycle of a Plugin, which has three states:resolved: the entrypoint pointing to the PluginSpec was imported and thePluginSpecinstance was createdinit: thePluginFactoryof thePluginSpecwas successfully invokedloaded: theloadmethod of thePluginwas successfully invokedLoading PluginsAt run time, aPluginManageruses aPluginFinderthat in turn uses stevedore to scan the available entrypoints for things that look like aPluginSpec.
WithPluginManager.load(name: str)orPluginManager.load_all(), plugins within the namespace that are discoverable in entrypoints can be loaded.
If an error occurs at any state of the lifecycle, thePluginManagerinforms thePluginLifecycleListenerabout it, but continues operating.Discovering entrypointsTo build a source distribution and a wheel of your code with your plugins as entrypoints, simply runpython setup.py plugins sdist bdist_wheel.
If you don't have asetup.py, you can use the plux build frontend and runpython -m plux entrypoints.How it works:
For discovering plugins at build time, plux provides a custom setuptools commandplugins, invoked viapython setup.py plugins.
The command uses a specialPluginFinderthat collects from the codebase anything that can be interpreted as aPluginSpec, and creates from it a plugin index fileplux.json, that is placed into the.egg-infodistribution metadata directory.
When a setuptools command is used to create the distribution (e.g.,python setup.py sdist/bdist_wheel/...), plux finds theplux.jsonplugin index and extends automatically the list of entry points (collected into.egg-info/entry_points.txt).
Theplux.jsonfile becomes a part of the distribution, s.t., the plugins do not have to be discovered every time your distribution is installed elsewhere.
Discovering at build time also works when usingpython -m build, since it calls registered setuptools scripts.ExamplesTo build something using the plugin framework, you will first want to introduce a Plugin that does something when it is loaded.
And then, at runtime, you need a component that uses thePluginManagerto get those plugins.One class per pluginThis is the way we went withLocalstackCliPlugin. Every plugin class (e.g.,ProCliPlugin) is essentially a singleton.
This is easy, as the classes are discoverable as plugins.
Simply create a Plugin class with a name and namespace and it will be discovered by the build timePluginFinder.frompluximportPlugin# abstract case (not discovered at build time, missing name)classCliPlugin(Plugin):namespace="my.plugins.cli"defload(self,cli):self.attach(cli)defattach(self,cli):raiseNotImplementedError# discovered at build time (has a namespace, name, and is a Plugin)classMyCliPlugin(CliPlugin):name="my"defattach(self,cli):# ... attach commands to cli objectnow we need aPluginManager(which has a generic type) to load the plugins for us:cli=# ... needs to come from somewheremanager:PluginManager[CliPlugin]=PluginManager("my.plugins.cli",load_args=(cli,))plugins:List[CliPlugin]=manager.load_all()# todo: do stuff with the plugins, if you want/need#  in this example, we simply use the plugin mechanism to run a one-shot function (attach) on a load argumentRe-usable pluginsWhen you have lots of plugins that are structured in a similar way, we may not want to create a separate Plugin class
for each plugin. Instead we want to use the samePluginclass to do the same thing, but use several instances of it.
ThePluginFactory, and the fact thatPluginSpecinstances defined at module level are discoverable (inpired
bypluggy), can be used to achieve that.frompluximportPlugin,PluginFactory,PluginSpecimportimportlibclassServicePlugin(Plugin):def__init__(self,service_name):self.service_name=service_nameself.service=Nonedefshould_load(self):returnself.service_nameinconfig.SERVICESdefload(self):module=importlib.import_module("localstack.services.%s"%self.service_name)# suppose we define a convention that each service module has a Service class, like moto's `Backend`self.service=module.Service()defservice_plugin_factory(name)->PluginFactory:defcreate():returnServicePlugin(name)returncreate# discoverables3=PluginSpec("localstack.plugins.services","s3",service_plugin_factory("s3"))# discoverabledynamodb=PluginSpec("localstack.plugins.services","dynamodb",service_plugin_factory("dynamodb"))# ... could be simplified with convenience framework code, but the principle will stay the sameThen we could use thePluginManagerto build a SupervisorfrompluximportPluginManagerclassSupervisor:manager:PluginManager[ServicePlugin]defstart(self,service_name):plugin=self.manager.load(service_name)service=plugin.serviceservice.start()Functions as pluginswith the@plugindecorator, you can expose functions as plugins. They will be wrapped by the framework
intoFunctionPlugininstances, which satisfy both the contract of a Plugin, and that of the function.frompluginimportplugin@plugin(namespace="localstack.configurators")defconfigure_logging(runtime):logging.basicConfig(level=runtime.config.loglevel)@plugin(namespace="localstack.configurators")defconfigure_somethingelse(runtime):# do other stuff with the runtime objectpassWith a PluginManager viaload_all, you receive theFunctionPlugininstances, that you can call like the functionsruntime=LocalstackRuntime()forconfiguratorinPluginManager("localstack.configurators").load_all():configurator(runtime)Configuring your distributionIf you are building a python distribution that exposes plugins discovered by plux, you need to configure your projects build system so other dependencies creates theentry_points.txtfile when installing your distribution.For apyproject.tomltemplate this involves adding thebuild-systemsection:[build-system]requires=['setuptools','wheel','plux>=1.3.1']build-backend="setuptools.build_meta"# ...Installpip install pluxDevelopCreate the virtual environment, install dependencies, and run testsmake venv
make testRun the code formattermake formatUpload the pypi package using twinemake upload