Package: algolink
Ebonite is a machine learning lifecycle framework.
It allows you to persist your models and reproduce them (as services or in general).Installationpip install eboniteQuickstartBefore you start with Ebonite you need to have your model.
This could be a model from your favorite library (list of supported libraries is below) or
just a custom Python function working with typical machine learning data.importnumpyasnpdefclf(data):return(np.sum(a,axis=-1)>1).astype(np.int32)Moreover, your custom function can wrap a model from some library.
This gives you flexibility to use not only pure ML models but rule-based ones (e.g., as a service stub at project start)
and hybrid (ML with pre/postprocessing) ones which are often applied to solve real world problems.When a model is prepared you should create an Ebonite client.fromeboniteimportEboniteebnt=Ebonite.local()Then create a task and push your model object with some sample data.
Sample data is required for Ebonite to determine structure of inputs and outputs for your model.task=ebnt.get_or_create_task('my_project','my_task')model=task.create_and_push_model(clf,test_x,'my_clf')You are awesome! Now your model is safely persisted in a repository.Later on in other Python process you can load your model from this repository and do some wonderful stuff with it,
e.g., create a Docker image namedmy_servicewith an HTTP service wrapping your model.fromeboniteimportEboniteebnt=Ebonite.local()task=ebnt.get_or_create_task('my_project','my_task')model=client.get_model('my_clf',task)client.build_image('my_service',model)Check out examples (inexamplesdirectory) and documentation to learn more.Documentation… is availablehereExamples… are available in thisfolder.
Here are some of them:Jupyter Notebook guideScikit-learn guideTensorFlow 2.0etc.Supported libraries and repositoriesModelsyour arbitrary Python functionscikit-learnTensorFlow (1.x and 2.x)XGBoostLightGBMPyTorchCatBoostModel input / output dataNumPypandasimagesModel repositoriesin-memorylocal filesystemSQLAlchemyAmazon S3ServingFlaskaiohttpCreate an issue if you need support for something other than that!ContributingReadthisChangelogCurrent release candidate0.6.2 (2020-06-18)Minor bugfixes0.6.1 (2020-06-15)Deleted accidental debug ‘print’ call :/0.6.0 (2020-06-12)Prebuilt flask server images for faster image buildMore and better methods in Ebonite clientPipelines - chain Models methods into one Model-like objectsRefactioring of image and instance APIRework of pandas DatasetType: now with column types, even non-primitive (e.g. datetimes)Helper functions for stanalone docker build/runMinor bugfixes and features0.5.2 (2020-05-16)Fixed dependency inspection to include wrapper dependenciesFixed s3 repo to fail with subdirectoriesMore flexible way to add parameters for instance running (e.g. docker run arguments)Added new type of Requirement to represent unix packages - for example, libgomp for xgboostMinor tweaks0.5.1 (2020-04-16)Minor fixes and examples update0.5.0 (2020-04-10)Built Docker images and running Docker containers along with their metadata are now persisted in metadata repositoryAdded possibility to track running status of Docker container via Ebonite clientImplemented support for pushing built images to remote Docker registryImproved testing of metadata repositories and Ebonite client and fixed discovered bugs in themFixed bug with failed transactions not being rolled backFixed bug with serialization of complex models some component of which could not be pickledDecomposed model IO from model wrappersbytes are now used for binary datasets instead of file-like objectsEliminated build_model_flask_docker in favor of Server-driven abstractionSped up PickleModelIO by avoiding ModelAnalyzer calls for non-model objectsSped up Model.create by calling model methods with given input data just onceDataset types and model wrappers expose their runtime requirements0.4.0 (2020-02-17)Implemented asyncio-based server via aiohttp libraryImplemented support for Tensorflow 2.x modelsChanged default type of base python docker image to “slim”Added ‘description’ and ‘params’ fields to Model. ‘description’ is a text field and ‘params’ is a dict with arbitrary keysFixed bug with building docker image with different python version that the Model was created with0.3.5 (2020-01-31)Fixed critical bug with wrapper_meta0.3.4 (2020-01-31)Fixed bug with deleting models from tasksSupport working with model meta without requiring installation of all model dependenciesAdded region argument for s3 repositorySupport for delete_model in Ebonite clientSupport for force flag in delete_model which deletes model even if artifacts could not be deleted0.3.3 (2020-01-10)Eliminated tensorflow warnings. Added more tests for providers/loaders. Fixed bugs in multi-model provider/builder.Improved documentationEliminate useless “which docker” check which fails on Windows hostsPerform redirect from / to Swagger API docs in Flask serverSupport for predict_proba method in ML modelDo not fix first dimension size for numpy arrays and torch tensorsSupport for Pytorch JIT (TorchScript) modelsBump tensorflow from 1.14.0 to 1.15.0Added more tests0.3.2 (2019-12-04)Multi-model interface bug fixes0.3.1 (2019-12-04)Minor bug fixes0.3.0 (2019-11-27)Added support for LightGBM modelsAdded support for XGBoost modelsAdded support for PyTorch modelsAdded support for CatBoost modelsAdded uwsgi server for flask containers0.2.1 (2019-11-19)Minor bug fixes0.2.0 (2019-11-14)First release on PyPI.