Package: mdbh
mdbh - A MongoDB helper collection to use with Sacred and Omniboard.This repository holds mostly two purposes:First, it provides a Python module together with some CLI scripts to ease the
usage of MongoDB together withSacredandOmniboard,
filling a low-level gap.
WhereasOmniboardis well suited
to  quickly explore data and compare Sacred experiments, it is not meant for
more complex data visualization and low-level database access.
This can for example be useful when preparing print-quality plots.Second, it provides aWikito collect guidelines on how to use Sacred with MongoDB, Omniboard and mdbh.
In particular, a multi-user, multi-database setup with password restriction and controlled
read/write access to multiple databases is provided. This Wiki is not meant
to be exhaustive, but shall get you started with your own setup.Note:This is still somewhat under development.[[TOC]]InstallationInstall via PyPi using pippipinstallmdbhSetupThe MongoDB instance configuration is done using one (or multiple) configuration
files which simply store the server IP, port and possible the username, password
and authentication methods and database names.
See theexamplesfolder for an example.By default, it is assumed the this configuration
file can be found under~/.mongo.confThis config file is central to the use of mdbh. As this config file might
contain userdata, make sure only the current user has read access to it,
e.g. runchmod400.mongo.confUse CasesSome use cases of mdbh are as follows:MongoDB URI creationOften times, services or modules (such as the MongoObserver in Sacred)
need a URI to connect to the MongoDB following theofficial format specification.
However, this often results in boilerplate code and, in the case of
user authentification, security risks as the username and/or password have
to be specified in the corresponding code.To this end, mdbh provides theget_urifunction which creates the connection URI based on the.mongo.conffile
and a specific database name.Sacred Experiment queryTo query and manipulate data that has been logged to the MongoDB instance
via Sacred, mdbh provides several functions, the most comprehensive one beingget_dataframewhich returns a PandasDataFrameobject created from the specified
Sacred database and possible run IDs. It conveniently combines configuration information
as well as logged metrics of all experiments in the specified database.
Using theDataFrame, complex data aggregation and/or plotting
(e.g. viaSeaborn) is quite comfortable.Sacred Experiment artifact retrievalTo retrieve artifacts from a Sacred experiments, mdbh provides theget_artifactfunction which downloads artifacts from the MongoDB and employs caching
for optimized re-use of artifacts across different applications.
Alternatively, artifacts can also be resolved for the full DataFrame
viaresolve_artifactswhich might however be slow as all artifacts are downloaded.MDBH configurationEnvironment variables, such as the cache directory, can be manipulated during
runtime via theenvironmodule.MongoDB Setup and OmniboardSee the correspondingOmniboard Wiki entryto see how mbdh can ease the use with Omniboard.See the correspondingMongoDB Wiki entryto see how to properly setup a MobgoDB instance.CachingBy default, MDBH caches downloaded artifacts, obtained viamdbh.get_artifact().
Optionally, DataFrames obtained viamdbh.get_dataframe()can be cached
locally using thecache=Trueoption. Please refer to the functions'
documentation for details.By default, the files are cached to a temporary folder obtained usingtempfile.gettempdir(), which for Linux usually defaults to/tmp. Hence,
the cache is not persistent across multiple boots. To make the cache persistent,
use a custom cache directory, for example:importmdbhmdbh.environ.set_cache_dir('~/.mdbh/cache')The environment variables are updated during runtime, reloading of the module
is hence not necessary.