Package: pypastry
PyPastry - the opinionated machine learning experimentation frameworkPyPastry is a framework for developers and data scientists to run
machine learning experiments. We enable you to:Iterate quickly. The more experiments you do, the more likely you
are to find something that works well.Experiment correctly and consistently. Anything else is not really
an experiment, is it?Make experiments reproducible. That means keeping track of your
code state and results.Experiment locally. None of that Spark rubbish.Use standard tools. Everything is based on Scikit-learn, Pandas and Git.Quick startPyPastry requires python 3.5 or greater.> pip install pypastry==0.2.0
> pastry init pastry-test
> cd pastry-test
> pastry run -m "First experiment"
Got dataset with 10 rows
   Git hash Dataset hash            Run start                   Model          Score Duration (s)
0  aa87ce62     71e8f4fd  2019-08-28 06:39:07  DecisionTreeClassifier  0.933 ± 0.067         0.03The commandpastry initcreates a file calledpie.pyin thepastry-testdirectory. If you open
that up, you should see some code. The important bit is:def get_experiment():
    dataset = pd.DataFrame({
        'feature': [1, 0, 1, 1, 0, 0, 1, 1, 0, 1],
        'class': [True, False, True, True, False, False, True, True, False, False],
    })
    predictor = DecisionTreeClassifier()
    cross_validator = StratifiedKFold(n_splits=5)
    scorer = make_scorer(f1_score)
    label_column = 'class'
    return Experiment(dataset, label_column, predictor, cross_validator, scorer)This returns anExperimentinstance that specifies how the experiment should be run. An experiment
consists of:dataset: a PandasDataFramewhere each row is an instance to be used in the experiment.label_column: the name of the column indatasetthat contains the label we wish to predict.predictor: a Scikit-learn predictor, e.g. a classifier, regressor orPipelineobject.cross_validator: a Scikit-learn cross validator that specifies how the data should be split
up when running the experiment.scorera Scikit-learn scorer that will be used as an indication of how well the classifier has
learnt to generate predictions.When you typepastry run, PyPastry does this:Splitsdatasetinto one or more train and test sets.For each train and test set, it trains thepredictoron the train set and generate predictions
on the test set, and computes the score on the test set using thescorer.Generates a results file in JSON format and stores it in a folder calledresultsOutputs the results of the experiment.Your repo has to be clean (no (un)staged changes) for experiment to run. If you want to use dirty repo, you can with calling pypsatry with force flag-f. However, results will not be possible to bond with exact code state.The results includes:Git hash: the commit identifier of the code used to run the experiment. There might be"dirty_"prefix indicating that unclean repo was used with this experiment. The hash belongs to the latest commit, however, the information about (un)staged changes is lost.Git summary: A summary noteDataset hash: a hash generated from the dataset that will change if the dataset changes.Run start: the time that the experiment run startedModel: the name of thepredictorclass usedScore: the mean ± the standard error in the mean, computed over the different folds generated
by thecross_validator.Duration: how long the experiment took to run, in seconds.ContributingPyPastry is at an early stage so there's plenty to do and we'd love to have your contribution.Check out the issues for a list of things that need doing and post a comment if you'd like to take
something on.If you have an idea for something you'd like to do, create an issue.Runpython -m pytestin the project root to run all tests.Thanks for using PyPastry!