Package: dfds-ds-toolbox
IntroductionThis repo is intended to contain a packaged toolbox of some neat,
frequently-used data science code snippets and functions. The intention is that
the classes should be compatible with thesklearnlibrary.Have a look athttps://dfds-ds-toolbox.readthedocs.iofor user guide.Already implemented:Model selector for regression and classification problemsProfiling tool for generating stats files of the execution time of a functionTo be implemented in the future:PreprocessingImbalanced datasetsOutlier detection & handlingMissing value imputationFeature generationBinningType variables, create multiple featuresTimestamp, seasonality variablesObject: onehot, grouping, etc.Performance analysis (plots, summary, error analysis)More ideas might arise in the future and should be added to the list.A guide on how to install the package and some working examples of how to use
the classes can be found in later sections.Getting StartedInstall locallyWe use poetry as the package manager and build tool. Make sure you have poetry
installed locally, then runpoetryinstallRun tests to see everything workingpoetryrunpytestInstall this library in another repoMake sure your virtual environment is activated, then install the required
packagespython-mpipinstall--upgradepipIf you want to install the packagedfds_ds_toolboxversion 0.8.0, you should
runpipinstalldfds_ds_toolbox==0.8.0VersionsSee changelog atGitHub.ContributeWe want this library to be useful across many data science projects. If you have
some standard utilities that you keep using in your projects, please add them
here and make a PR.Releasing a new versionWhen you want to release a new version of this library toPyPI, there is a few steps you must
follow.Update the version insetup.py. We followSemantic Versioning, so think about if there is any
breaking changes in your release when you increment the version.Draft a new release inGithub. You can
follow this link or click the "Draft a new release button" on the "releases"
page.Here you must add a tag in the form "v", for example "v0.9.2".
The title should be the same as the tag.Add release notes. The easiest is to use the button "Auto-generate release
notes". That will pull titles of completed pull requests. Modify as
needed.Click "Publish release". That will start aGithub Actionthat
will build the package and upload to PyPI. It will also build the
documentation website.DocumentationWebsiteThe full documentation of this package is available athttps://dfds-ds-toolbox.readthedocs.ioTo build the documentation locally run:pipinstall-rdocs/requirements.txtcddocs/
sphinx-apidoc-o.../dfds_ds_toolbox/../*tests*
makehtmlNow, you can open the documentation site indocs/_build/index.html.StyleWe are using GooglesPython style guideconvention for docstrings. This allows us to make an up-to-date documentation
website for the package.In short, every function should have a short one-line description, optionally a
longer description afterwards and a list of parameters. For exampledefexample_function(some_parameter:str,optional_param:int=None)->bool:"""This function does something super smart.Here I will dive into more detail about the smart things.I can use several lines for that.Args:some_parameter: Name of whateveroptional_param: Number of widgets or something. Only included when all the starts align.Returns:An indicator describing if something is true."""There are many other style issues that we can run into, but if you follow the
Google style guide, you will probably be fine.ExamplesTo show the intended use and outcome of some of the included methods, we have
included a gallery of plots inexamples/. To make a new example create a new
file and name it something likeplot_<whatever>.py. Start this file with a
docstring, for example"""Univariate plots================For a list of features separate in bins and analysis the target distribution in both Train and Test"""and after this add the python code needed to create the example plot.