Package: jackdaw-ml
Jackdaw MLJackdaw is a framework designed to help Data Scientists save, load, and deploy Machine Learning models in a
simple and robust way.Unlike other frameworks, sharing a model only requires two short lines of code.@find_artefactsScan model code, finding model artefacts and variables. Jackdaw supports PyTorch, Tensorflow, XGBoost, and LightGBM
out of the box, but is trivial for users to expand to other frameworks.jackdaw_ml.savesModel artefacts are saved to either local or remote storage, and the user is provided with a Model ID.
Users can store the ID for use in deployment, or search for the model by its name, metrics, Git branch, etc.A Model can be as simple as a stored number, or as complex as a combination of frameworks. Regardless of complexity,
Jackdaw aims to make it simple to share your models with other applications, other colleagues, or other companies.Documentation is baked into the repository, and is availablehere.Setup - Working LocallyJackdaw is available onPyPiand can be installed via pip;>>>pipinstalljackdaw_mlAlpha - Limited Windows & Mac SupportWhile Jackdaw is in Alpha, one of the libraries it relies upon - artefactlink - only supports Windows and Mac OS/X for Python 3.10. Linux support is available for 3.8, 3.9, and 3.10.Setup - Sharing Models across EnvironmentsTo share items across multiple computers, you'll eventually need an account with ShareableAI.For now, you just need your API Token. If you don't have a token, reach out tolissa@shareablei.comand they'll ping you one.Roadmap - Future FeaturesView our Public Roadmap hereGetting StartedExample by FrameworkSKLearnLightGBMXGBoostPyTorchTensorflowDARTsExampleThe core magic of Jackdaw is within the@artefactsand@find_artefactsdecorators.@artefactsallows you to list what should be saved on a Model.@find_artefactswill detect what should be saved based
on a whole host of common frameworks. Combining the two is a powerful way of ensuring complex models can be saved easily.fromjackdaw_ml.artefact_decoratorimportfind_artefactsfromjackdaw_ml.traceimporttrace_artefactsfromjackdaw_mlimportsavesimportlightgbmaslgbimportnumpyasnp@find_artefacts()classBasicLGBWrapper:model:lgb.Booster# The LightGBM model isn't defined yet, but we can discover it via annotations.#   LightGBM is Pickle-safe, so it's serialised via the PickleSerialiser by default.# >>> trace_artefacts(BasicLGBWrapper())# <class '__main__.BasicLGBWrapper'>{#         (model) [<class 'jackdaw_ml.serializers.pickle.PickleSerializer'>]# }defexample_data()->lgb.Dataset:data=np.random.rand(500,10)# 500 entities, each contains 10 featureslabel=np.random.randint(2,size=500)# binary targetreturnlgb.Dataset(data,label=label)# Create a new Modelmodel=BasicLGBWrapper()# Modify the modelmodel.model=lgb.train({},example_data())# Save the modelmodel_id=saves(model)Saving RemotelySaving and loading items from ShareableAI servers, rather than locally, can be achieved by providing an API key alongside the call toartefacts. If you'd like to test this, please follow the setup for Sharing Models Remotely.fromjackdaw_ml.artefact_decoratorimportartefactsfromjackdaw_ml.serializers.pickleimportPickleSerializerfromjackdaw_ml.artefact_endpointimportArtefactEndpoint@artefacts({PickleSerializer:["x"]},endpoint=ArtefactEndpoint.remote('MyAPIKey'))classMyModel:def__init__(self):self.x=3We'll be expanding this to allow for sharing items between other usersverysoon, so keep an eye onCorvusto know more.