Repository: JacksonBurns/astartes (language: python)

Issue:
pyOpenSci Initial Review Cleanup
## Feedback from Initial Submission to pyOpenSci

In the meanwhile I have a first question for you, if you don't mind.
While the statistical side of the package is clear and answers to a well known need of the scikit-learn user community, I would like to read a bit more about the package connections with chemical engineering. I guess your scientific application was the motivation for this development in the first place and in the documentation the two aspects look quite unrelated. Perhaps a small pedagogical section about chemical applications in the documentation will also be helpful? Thanks!

_Originally posted by @cmarmo in https://github.com/pyOpenSci/software-submission/issues/120#issuecomment-1631493213_

We need to clarify the applications of `astartes` to chemical engineering _in particular_ rather than just the importance in terms of statistics and ML, broadly speaking. As mentioned by @kspieks offline there is some content in the JOSS paper that could be pulled from to flesh this out.

I would suggest that we make a separate markdown document, like the 'sklearn to astartes' one and reference it in the main README.
            


Discussion / hints:
## Additional Comments:
(see [here](https://github.com/pyOpenSci/software-submission/issues/120#issuecomment-1635069583) for full explanations)

### References

Many references in the README are paywalled, try and find open-access versions. Some references are also missing, include them. Also make sure that the `References` column of the `Included Sampling Algorithms` table clarifies what the link actually is a reference to (documentation, a paper, etc.).

### Formatting Fixes

External links with backquotes are breaking in the Sphinx rendered documentation.

### Documentation

Add further examples of some of the backend, and how users could leverage other functionality implemented in `astartes` besides just `train_(val_)test_split`.

### Version Attribute

Specify a `__version__` attribute, as well as directions for how to retrieve it. Can use [this](https://packaging.python.org/en/latest/guides/single-sourcing-package-version/) to keep compatibility with `pyproject.toml`:
```toml
[project]
name = "astartes"
dynamic = ["version"]

[tool.setuptools.dynamic]
version = {attr = "package.__version__"}
```
Which is compatible with both `from importlib.metadata import version` and `astartes.__version__` approaches.

Fix patch:
diff --git a/.github/workflows/gen_docs.yml b/.github/workflows/gen_docs.yml
index 37cb3aae..db581e2e 100644
--- a/.github/workflows/gen_docs.yml
+++ b/.github/workflows/gen_docs.yml
@@ -41,9 +41,13 @@ jobs:
           rm docs/README.rst || true
           rm sklearn_to_astartes.rst || true
           rm docs/sklearn_to_astartes.rst || true
+          rm CONTRIBUTING.rst || true
+          rm docs/CONTRIBUTING.rst || true
           m2r README.md
+          m2r CONTRIBUTING.md
           m2r sklearn_to_astartes.md
-          mv README.rst docs
+          mv README.rst 
+          mv CONTRIBUTING.rst docs
           mv sklearn_to_astartes.rst docs
           cd docs
           rm *.html *.doctree || true
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 506b1a2a..d8741771 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -1,24 +1,40 @@
 ## Contributing & Developer Notes
-Pull Requests, Bug Reports, and all Contributions are welcome! Please use the appropriate issue or pull request template when making a contribution.
+Pull Requests, Bug Reports, and all Contributions are welcome, encouraged, and appreciated!
+Please use the appropriate [issue](https://github.com/JacksonBurns/astartes/issues/new/choose) or [pull request](https://github.com/JacksonBurns/astartes/compare) template when making a contribution to help the maintainers get it merged quickly.
 
-We make use of [the GitHub Discussions page](https://github.com/JacksonBurns/astartes/discussions) to go over potential features to add. Please feel free to stop by if you are looking for something to develop or have an idea for a useful feature!
+We make use of [the GitHub Discussions page](https://github.com/JacksonBurns/astartes/discussions) to go over potential features to add.
+Please feel free to stop by if you are looking for something to develop or have an idea for a useful feature!
 
 When submitting a PR, please mark your PR with the "PR Ready for Review" label when you are finished making changes so that the GitHub actions bots can work their magic!
 
 ### Developer Install
 
-To contribute to the `astartes` source code, start by cloning the repository (i.e. `git clone git@github.com:JacksonBurns/astartes.git`) and then inside the repository run `pip install -e .[molecules,dev]`. This will set you up with all the required dependencies to run `astartes` and conform to our formatting standards (`black` and `isort`), which you can configure to run automatically in vscode [like this](https://marcobelo.medium.com/setting-up-python-black-on-visual-studio-code-5318eba4cd00#:~:text=Go%20to%20settings%20in%20your,%E2%80%9D%20and%20select%20%E2%80%9Cblack%E2%80%9D.).
+To contribute to the `astartes` source code, start by forking and then cloning the repository (i.e. `git clone git@github.com:YourUsername/astartes.git`) and then inside the repository run `pip install -e .[dev]`. This will set you up with all the required dependencies to run `astartes` and conform to our formatting standards (`black` and `isort`), which you can configure to run automatically in VSCode [like this](https://marcobelo.medium.com/setting-up-python-black-on-visual-studio-code-5318eba4cd00#:~:text=Go%20to%20settings%20in%20your,%E2%80%9D%20and%20select%20%E2%80%9Cblack%E2%80%9D.).
 
 > **Note**
-> Windows Powershell and MacOS Catalina or newer may complain about square brackets, so you will need to double quote the `molecules` command (i.e. `pip install "astartes[molecules,dev]"`)
+> Windows Powershell and MacOS Catalina or newer may complain about square brackets, so you will need to double quote the `molecules` command (i.e. `pip install "astartes[dev]"`)
 
-### Unit Testing
-All of the tests in `astartes` are written using the built-in python `unittest` module (to allow running without `pytest`) but we _highly_ recommend using `pytest`. To execute the tests from the `astartes` repository, simply type `pytest` after running the developer install (or alternately, `pytest -v` for a more helpful output).
+### Version Checking
+
+`astartes` uses `pyproject.toml` to specify all metadata _except_ the version, which is specified in `astartes/__init__.py` (via `__version__`) for backwards compatibility with Python 3.7.
+To check which version of `astartes` you have installed, you can run `python -c "import astartes; print(astartes.__version__)"` on Python 3.7 or `python -c "from importlib.metadata import version; version('astartes')" on Python 3.8 or newer.
+
+### Testing
+All of the tests in `astartes` are written using the built-in python `unittest` module (to allow running without `pytest`) but we _highly_ recommend using `pytest`.
+To execute the tests from the `astartes` repository, simply type `pytest` after running the developer install (or alternately, `pytest -v` for a more helpful output).
+On GitHub, we use actions to run the tests on every Pull Request and on a nightly basis (look in `.github/workflows` for more information).
+These tests include unit tests, functional tests, and regression tests.
 
 ### Adding New Samplers
 Adding a new sampler should extend the `abstract_sampler.py` abstract base class.
+Each subclass should override the `_sample` method with its own algorithm for data partitioning, and the constructor (`__init__.py`) perform any data validation.
+
+All samplers in `astartes` are classified as one of two types: extrapolative or interpolative.
+Extrapolative samplers work by clustering data into groups (which are then partitioned into train/validation/test to enforce extrapolation) whereas interpolative samplers provide an exact _order_ in which samples should be moved into the training set.
 
-It can be as simple as a passthrough to a another `train_test_split`, or it can be an original implementation that results in X and y being split into two lists. Take a look at `astartes/samplers/random_split.py` for a basic example!
+When actually implemented, this means that extrapolative samplers should set the `self._samples_clusters` attribute and interpolative samplers should set the `self._samples_idxs` attribute.
+
+New samplers can be as simple as a passthrough to another `train_test_split`, or it can be an original implementation that results in X and y being split into two lists. Take a look at `astartes/samplers/interpolation/random_split.py` for a basic example!
 
 After the sampler has been implemented, add it to `__init__.py` in in `astartes/samplers` and it will automatically be unit tested. Additional unit tests to verify that hyperparameters can be properly passed, etc. are also recommended.
 
@@ -71,6 +87,22 @@ If possible, we would like to also add an example Jupyter Notebook with any new
 
 Contact [@JacksonBurns](https://github.com/JacksonBurns) if you need assistance adding an existing workflow to `astartes`. If this featurization scheme requires additional dependencies to function, we may add it as an additional _extra_ package in the same way that `molecules` in installed.
 
+### The `train_val_test_split` Function
+`train_val_test_split` is the workhorse function of `astartes`.
+It is responsible for instantiating the sampling algorithm, partitioning the data into training, validation, and testing, and then returning the requested results while also keeping an eye on data types.
+Under the hood, `train_test_split` is just calling `train_val_test_split` with `val_size` set to `0.0`.
+For more information on how it works, check out the inline documentation in `astartes/main.py`.
+
+### Development Philosophy
+
+The developers of `astartes` prioritize (1) reproducibility, (2) flexibility, and (3) maintainability.
+ 1. All versions of `astartes` `1.x` should produce the same results across all platforms, so we have thorough unit and regression testing run on a continuous basis.
+ 2. We specify as _few dependencies as possible_ with the _loosest possible_ dependency requirements, which allows integrating `astartes` with other tools more easily.
+  - Dependencies which introduce a lot of requirements and/or specific versions of requirements are shuffled into the `extras_require` to avoid weighing down the main package.
+  - Compatibility with all versions of modern Python is achieved by not tightly specifying version numbers as well as by regression testing across all versions.
+ 3. We follow DRY (Don't Repeat Yourself) principles to avoid code duplication and decrease maintainence burden, have near-perfect test coverage, and enforce consistent formatting style in the source code.
+  - Inline comments are _critical_ for maintainability - at the time of writing, `astartes` has 1 comment line for every 2 lines of source code.
+
 ## JOSS Branch
 
 `astartes` corresponding JOSS paper is stored in this repository on a separate branch. You can find `paper.md` on the aptly named `joss-paper` branch. 
diff --git a/README.md b/README.md
index ed6e9d6b..1e43972d 100644
--- a/README.md
+++ b/README.md
@@ -12,8 +12,13 @@
   <img alt="PyPI - License" src="https://img.shields.io/github/license/JacksonBurns/astartes">
   <img alt="Test Status" src="https://github.com/JacksonBurns/astartes/actions/workflows/run_tests.yml/badge.svg?branch=main&event=schedule">
   <img alt="Reproduce Paper" src="https://github.com/JacksonBurns/astartes/actions/workflows/reproduce_paper.yml/badge.svg?branch=main&event=schedule">
+  <a href="https://doi.org/10.5281/zenodo.8147205"><img src="https://zenodo.org/badge/DOI/10.5281/zenodo.8147205.svg" alt="DOI"></a>
 </p>
 
+## Online Documentation
+Follow [this link](https://JacksonBurns.github.io/astartes/) for a nicely-rendered version of this README along with additional tutorials for [moving from train_test_split in sklearn to astartes](https://jacksonburns.github.io/astartes/sklearn_to_astartes.html).
+Keep reading for a installation guide and links to tutorials!
+
 ## Installing `astartes`
 We recommend installing `astartes` within a virtual environment, using either `venv` or `conda` (or other tools) to simplify dependency management. Python versions 3.7, 3.8, 3.9, 3.10, and 3.11 are supported on all platforms.
 
@@ -30,8 +35,8 @@ Note that this package _does not_ include built-in support for featurizing molec
 
 To install `astartes` from source, see the [Contributing & Developer Notes](#contributing--developer-notes) section.
 
-## Using `astartes`
-`astartes` is designed as a drop-in replacement for `sklearn`'s `train_test_split` function. To switch to `astartes`, change `from sklearn.model_selection import train_test_split` to `from astartes import train_test_split`.
+## Quick Start
+`astartes` is designed as a drop-in replacement for `sklearn`'s `train_test_split` function (see the [sklearn documentation](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html)). To switch to `astartes`, change `from sklearn.model_selection import train_test_split` to `from astartes import train_test_split`.
 
 Like `sklearn`, `astartes` accepts any iterable object as `X`, `y`, and `labels`.
 Each will be converted to a `numpy` array for internal operations, and returned as a `numpy` array with limited exceptions: if `X` is a `pandas` `DataFrame`, `y` is a `Series`, or `labels` is a `Series`, `astartes` will cast it back to its original type including its index and column names.
@@ -39,7 +44,7 @@ Each will be converted to a `numpy` array for internal operations, and returned
 > **Note**
 > The developers recommend passing `X`, `y`, and `labels` as `numpy` arrays and handling the conversion to and from other types explicity on your own. Behind-the-scenes type casting can lead to unexpected behavior!
 
-By default, `astartes` will split data randomly. Additionally, a variety of algorithmic sampling approaches can be used by specifying the `sampler` argument to the function:
+By default, `astartes` will split data randomly. Additionally, a variety of algorithmic sampling approaches can be used by specifying the `sampler` argument to the function (see the [Table of Implemented Samplers](#implemented-sampling-algorithms) for a complet list of options and their corresponding references):
 
 ```python
 X_train, X_test, y_train, y_test = train_test_split(
@@ -49,46 +54,116 @@ X_train, X_test, y_train, y_test = train_test_split(
 )
 ```
 
-### Paper
-For a comprehensive walkthrough of the theory and implementation of `astartes`, follow [this link](https://github.com/JacksonBurns/astartes/raw/joss-paper/Burns-Spiekermann-Bhattacharjee_astartes.pdf) to read the companion paper.
+That's all you need to get started with `astartes`! The next sections include more examples and some demo notebooks you can try in your browser.
 
 ### Example Notebooks
 
 Click the badges in the table below to be taken to a live, interactive demo of `astartes`:
 
-| Demo | Link |
-|:---:|---|
-| Using `train_val_test_split` with the `sklearn` example datasets | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/JacksonBurns/astartes/main?labpath=examples%2Ftrain_val_test_split_sklearn_example%2Ftrain_val_test_split_example.ipynb) |
-| Comparing Sampling Algorithms with Fast Food | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/JacksonBurns/astartes/main?labpath=examples%2Fsplit_comparisons%2Fsplit_comparisons.ipynb) |
-| Cheminformatics sample set partitioning with `astartes` | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/JacksonBurns/astartes/main?labpath=examples%2Fbarrier_prediction_with_RDB7%2FRDB7_barrier_prediction_example.ipynb) |
-| Comparing partitioning approaches for alkanes | [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/JacksonBurns/astartes/main?labpath=examples%2Fmlpds_2023_astartes_demo%2Fmlpds_2023_demo.ipynb) |
+| Demo | Topic | Link |
+|:---:|---|---|
+| Comparing Sampling Algorithms with Fast Food | Visual representations of how different samplers affect data partitioning | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JacksonBurns/astartes/blob/main/examples/split_comparisons/split_comparisons.ipynb) |
+| Using `train_val_test_split` with the `sklearn` example datasets | Demonstrating how witholding a test set with `train_val_test_split` can impact performance | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JacksonBurns/astartes/blob/main/examples/train_val_test_split_sklearn_example/train_val_test_split_example.ipynb) |
+| Cheminformatics sample set partitioning with `astartes` | Extrapolation vs. Interpolation impact on cheminformatics model accuracy | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JacksonBurns/astartes/blob/main/examples/barrier_prediction_with_RDB7/RDB7_barrier_prediction_example.ipynb) |
+| Comparing partitioning approaches for alkanes | Visualizing how sampler impact model performance with simple chemicals | [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JacksonBurns/astartes/blob/main/examples/mlpds_2023_astartes_demonstration/mlpds_2023_demo.ipynb) |
 
 To execute these notebooks locally, clone this repository (i.e. `git clone https://github.com/JacksonBurns/astartes.git`), navigate to the `astartes` directory, run `pip install .[demos]`, then open and run the notebooks in your preferred editor.
+You do _not_ need to execute the cells prefixed with `%%capture` - they are only present for compatibility with Google Colab.
+
+### Withhold Testing Data with `train_val_test_split`
+For rigorous ML research, it is critical to withhold some data during training to use a `test` set.
+The model should _never_ see this data during training (unlike the validation set) so that we can get an accurate measurement of its performance.
+
+With `astartes` performing this three-way data split is readily available with `train_val_test_split`:
+```python
+from astartes import train_val_test_split
+
+X_train, X_val, X_test = train_val_test_split(X, sampler = 'sphere_exclusion')
+```
+You can now train your model with `X_train`, optimize your model with `X_val`, and measure its performance with `X_test`.
+
+### Evaluate the Impact of Splitting Algorithms
+For data with many features it can be difficult to visualize how different sampling algorithms change the distribution of data into training, validation, and testing like we do in some of the demo notebooks.
+To aid in analyzing the impact of the algorithms, `astartes` provides `generate_regression_results_dict`.
+This function allows users to quickly evaluate the impact of different splitting techniques on any model supported by `sklearn`. All results are stored in a dictionary format and can be displayed in a neatly formatted table using the optional `print_results` argument.
+
+```python
+from sklearn.svm import LinearSVR
+
+from astartes.utils import generate_regression_results_dict
+
+sklearn_model = LinearSVR()
+results_dict = generate_regression_results_dict(
+    sklearn_model,
+    X,
+    y,
+    print_results=True,
+)
+
+         Train       Val      Test
+----  --------  --------  --------
+MAE   1.41522   3.13435   2.17091
+RMSE  2.03062   3.73721   2.40041
+R2    0.90745   0.80787   0.78412
+
+```
+
+### Access Sampling Algorithms Directly
+The sampling algorithms implemented in `astartes` can also be directly accessed and run if it is more useful for your applications.
+In the below example, we import the Kennard Stone sampler, use it to partition a simple array, and then retrieve a sample.
+```python
+from astartes.samplers.interpolation import KennardStone
+
+kennard_stone = KennardStone([[1, 2], [3, 4], [5, 6]])
+first_2_samples = kennard_stone.get_sample_idxs(2)
+```
+All samplers in `astartes` implement a `_sample()` method that is called by the constructor (i.e. greedily) and either a `get_sampler_idxs` or `get_cluster_idxs` for interpolative and extrapolative samplers, respectively.
+For more detail on the implementaiton and design of samplers in `astartes`, see the [Developer Notes](#contributing--developer-notes) section.
+
+## Theory and Application of `astartes`
+This section of the README details some of the theory behind why the algorithms implemented in `astartes` are important and some motivating examples.
+For a comprehensive walkthrough of the theory and implementation of `astartes`, follow [this link](https://github.com/JacksonBurns/astartes/raw/joss-paper/Burns-Spiekermann-Bhattacharjee_astartes.pdf) to read the companion paper (freely available and hosted here on GitHub).
+
+> **Note**
+> We reference open-access publications wherever possible. For articles locked behind a paywall (denoted with :small_blue_diamond:), we instead suggest reading [this Wikipedia page](https://en.wikipedia.org/wiki/Sci-Hub) and absolutely __not__ attempting to bypass the paywall.
 
 ### Rational Splitting Algorithms
-While much machine learning is done with a random choice between training/validation/test data, an alternative is the use of so-called "rational" splitting algorithms. These approaches use some similarity-based algorithm to divide data into sets. Some of these algorithms include Kennard-Stone, minimal test set dissimilarity, and sphere exclusion algorithms [as discussed by Tropsha et. al](https://pubs.acs.org/doi/pdf/10.1021/ci300338w) as well as the OptiSim as discussed in [Applied Chemoinformatics: Achievements and Future Opportunities](https://www.wiley.com/en-us/Applied+Chemoinformatics%3A+Achievements+and+Future+Opportunities-p-9783527806546). Some clustering-based splitting techniques have also been incorporated, such as [DBSCAN](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1016.890&rep=rep1&type=pdf).
+While much machine learning is done with a random choice between training/validation/test data, an alternative is the use of so-called "rational" splitting algorithms.
+These approaches use some similarity-based algorithm to divide data into sets.
+Some of these algorithms include Kennard-Stone ([Kennard & Stone](https://www.tandfonline.com/doi/abs/10.1080/00401706.1969.10490666) :small_blue_diamond:), Sphere Exclusion ([Tropsha et. al](https://pubs.acs.org/doi/pdf/10.1021/ci300338w) :small_blue_diamond:),as well as the OptiSim as discussed in [Applied Chemoinformatics: Achievements and Future Opportunities](https://www.wiley.com/en-us/Applied+Chemoinformatics%3A+Achievements+and+Future+Opportunities-p-9783527806546) :small_blue_diamond:.
+Some clustering-based splitting techniques have also been incorporated, such as [DBSCAN](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1016.890&rep=rep1&type=pdf).
 
-There are two broad categories of sampling algorithms implemented in `astartes`: extrapolative and interpolative. The former will force your model to predict on out-of-sample data, which creates a more challenging task than interpolative sampling. See the table below for all of the sampling approaches currently implemented in `astartes`, as well as the hyperparameters that each algorithm accepts (which are passed in with `hopts`) and a helpful reference for understanding how the hyperparameters work. Note that `random_state` is defined as a keyword argument in `train_test_split` itself, even though these algorithms will use the `random_state` in their own work. Do not provide a `random_state` in the `hopts` dictionary - it will be overwritten by the `random_state` you provide for `train_test_split` (or the default if none is provided).
+There are two broad categories of sampling algorithms implemented in `astartes`: extrapolative and interpolative.
+The former will force your model to predict on out-of-sample data, which creates a more challenging task than interpolative sampling.
+See the table below for all of the sampling approaches currently implemented in `astartes`, as well as the hyperparameters that each algorithm accepts (which are passed in with `hopts`) and a helpful reference for understanding how the hyperparameters work.
+Note that `random_state` is defined as a keyword argument in `train_test_split` itself, even though these algorithms will use the `random_state` in their own work.
+Do not provide a `random_state` in the `hopts` dictionary - it will be overwritten by the `random_state` you provide for `train_test_split` (or the default if none is provided).
 
 #### Implemented Sampling Algorithms
 
 | Sampler Name | Usage String | Type | Hyperparameters | Reference | Notes |
 |:---:|---|---|---|---|---|
-| Random | 'random' | Interpolative | `shuffle` | [`sklearn train_test_split`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html) | This sampler is a direct passthrough to `sklearn`'s `train_test_split`, though it does not currently reproduce splits identically. |
-| Kennard-Stone | 'kennard_stone' | Interpolative | `metric` | [Kennard & Stone](https://www.tandfonline.com/doi/abs/10.1080/00401706.1969.10490666) | Euclidian distance is used by default, as described in the original paper. |
-| Sample set Partitioning based on joint X-Y distances (SPXY) | 'spxy' | Interpolative | `distance_metric` | Saldhana et. al [original paper](https://www.sciencedirect.com/science/article/abs/pii/S003991400500192X) | Extension of Kennard Stone that also includes the response when sampling distances. |
-| Scaffold | 'scaffold' | Extrapolative | `include_chirality` | [Bemis-Murcko Scaffold](https://pubs.acs.org/doi/full/10.1021/jm9602928) as implemented in RDKit | This sampler requires SMILES strings as input (use the `molecules` subpackage) |
+| Random | 'random' | Interpolative | `shuffle` | [sklearn train_test_split](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html) Documentation | This sampler is a direct passthrough to `sklearn`'s `train_test_split`. |
+| Kennard-Stone | 'kennard_stone' | Interpolative | `metric` | Original Paper by [Kennard & Stone](https://www.tandfonline.com/doi/abs/10.1080/00401706.1969.10490666) :small_blue_diamond: | Euclidian distance is used by default, as described in the original paper. |
+| Sample set Partitioning based on joint X-Y distances (SPXY) | 'spxy' | Interpolative | `distance_metric` | Saldhana et. al [original paper](https://www.sciencedirect.com/science/article/abs/pii/S003991400500192X) :small_blue_diamond: | Extension of Kennard Stone that also includes the response when sampling distances. |
+| Scaffold | 'scaffold' | Extrapolative | `include_chirality` | [Bemis-Murcko Scaffold](https://pubs.acs.org/doi/full/10.1021/jm9602928) :small_blue_diamond: as implemented in RDKit | This sampler requires SMILES strings as input (use the `molecules` subpackage) |
 | Sphere Exclusion | 'sphere_exclusion' | Extrapolative | `metric`, `distance_cutoff` | _custom implementation_ | Variation on Sphere Exclusion for arbitrary-valued vectors. |
-| Time Based | 'time_based' | Extrapolative | _none_ | [Chen et al.](https://pubs.acs.org/doi/full/10.1021/ci200615h), [Sheridan, R. P](https://pubs.acs.org/doi/full/10.1021/ci400084k), [Feinberg et al.](https://pubs.acs.org/doi/full/10.1021/acs.jmedchem.9b02187), [Struble et al.](https://pubs.rsc.org/en/content/articlehtml/2020/re/d0re00071j)  | This sampler requires `labels` to be an iterable of either date or datetime objects. |
+| Time Based | 'time_based' | Extrapolative | _none_ | Papers using Time based splitting: [Chen et al.](https://pubs.acs.org/doi/full/10.1021/ci200615h) :small_blue_diamond:, [Sheridan, R. P](https://pubs.acs.org/doi/full/10.1021/ci400084k) :small_blue_diamond:, [Feinberg et al.](https://pubs.acs.org/doi/full/10.1021/acs.jmedchem.9b02187) :small_blue_diamond:, [Struble et al.](https://pubs.rsc.org/en/content/articlehtml/2020/re/d0re00071j) | This sampler requires `labels` to be an iterable of either date or datetime objects. |
 | Optimizable K-Dissimilarity Selection (OptiSim) | 'optisim' | Extrapolative | `n_clusters`, `max_subsample_size`, `distance_cutoff` | _custom implementation_ | Variation on [OptiSim](https://pubs.acs.org/doi/10.1021/ci025662h) for arbitrary-valued vectors. |
 | K-Means | 'kmeans' | Extrapolative | `n_clusters`, `n_init` | [`sklearn KMeans`](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html) | Passthrough to `sklearn`'s `KMeans`. |
-| Density-Based Spatial Clustering of Applications with Noise (DBSCAN) | 'dbscan' | Extrapolative | `eps`, `min_samples`, `algorithm`, `metric`, `leaf_size` | [`sklearn DBSCAN`](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html) | Passthrough to `sklearn`'s `DBSCAN`. |
+| Density-Based Spatial Clustering of Applications with Noise (DBSCAN) | 'dbscan' | Extrapolative | `eps`, `min_samples`, `algorithm`, `metric`, `leaf_size` | [`sklearn DBSCAN`](https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html) Documentation| Passthrough to `sklearn`'s `DBSCAN`. |
 | Minimum Test Set Dissimilarity (MTSD) | ~ | ~ | _upcoming in_ `astartes` _v1.x_ | ~ | ~ |
 | Restricted Boltzmann Machine (RBM) | ~ | ~ | _upcoming in_ `astartes` _v1.x_ | ~ | ~ |
 | Kohonen Self-Organizing Map (SOM) | ~ | ~ | _upcoming in_ `astartes` _v1.x_ | ~ | ~ |
 | SPlit Method | ~ | ~ | _upcoming in_ `astartes` _v1.x_ | ~ | ~ |
 
-### Using the `astartes.molecules` Subpackage
+### Domain-Specific Applications
+Below are some field specific applications of `astartes`. Interested in adding a new sampling algorithm or featurization approach? See [`CONTRIBUTING.md`](./CONTRIBUTING.md).
+
+#### Chemical Data and the `astartes.molecules` Subpackage
+Machine Learning is enormously useful in chemistry-related fields due to the high-dimensional feature space of chemical data.
+To properly apply ML to chemical data for inference _or_ discovery, it is important to know a model's accuracy under the two domains.
+To simplify the process of partitioning chemical data, `astartes` implements a pre-built featurizer for common chemistry data formats.
 After installing with `pip install astartes[molecules]` one can import the new train/test splitting function like this: `from astartes.molecules import train_test_split_molecules`
 
 The usage of this function is identical to `train_test_split` but with the addition of new arguments to control how the molecules are featurized:
@@ -117,65 +192,44 @@ train_test_split_molecules(
 )
 ```
 
-To see a complete example of using `train_test_split_molecules` with actual chemical data, take a look in the `examples` directory.
+To see a complete example of using `train_test_split_molecules` with actual chemical data, take a look in the `examples` directory and the brief [companion paper](https://github.com/JacksonBurns/astartes/raw/joss-paper/Burns-Spiekermann-Bhattacharjee_astartes.pdf).
 
-Configuration options for the featurization scheme can be found in the documentation for [`AIMSim`](https://vlachosgroup.github.io/AIMSim/README.html#currently-implemented-fingerprints) though most of the critical configuration options are shown above.
+Configuration options for the featurization scheme can be found in the documentation for [AIMSim](https://vlachosgroup.github.io/AIMSim/README.html#currently-implemented-fingerprints) though most of the critical configuration options are shown above.
 
-### Reproducibility
+## Reproducibility
 `astartes` aims to be completely reproducible across different platforms, Python versions, and dependency configurations - any version of `astartes` v1.x should result in the _exact_ same splits, always.
 To that end, the default behavior of `astartes` is to use `42` as the random seed and _always_ set it.
 Running `astartes` with the default settings will always produce the exact same results.
 We have verified this behavior on Debian Ubuntu, Windows, and Intel Macs from Python versions 3.7 through 3.11 (with appropriate dependencies for each version).
 
-#### Known Reproducibility Limitations
+### Known Reproducibility Limitations
+Inevitably external dependencies of `astartes` will introduce backwards-incompatible changes.
+We continually run regression tests to catch these, and will list all _known_ limitations here:
+ - `sklearn` v1.3.0 introduced backwards-incompatible changes in the `KMeans` sampler that changed how the random initialization affects the results, even given the same random seed. Different version of `sklearn` will affect the performance of `astartes` and we recommend including the exact version of `scikit-learn` and `astartes` used, when applicable.
 
 > **Note**
 > We are limited in our ability to test on M1 Macs, but from our limited manual testing we achieve perfect reproducbility in all cases _except occasionally_ with `KMeans` on Apple silicon.
-It has produced _slightly_ different results between platforms regardless of `random_state`, with up to two clusters being assigned differently resulting in data splits which are >99% identical.
 `astartes` is still consistent between runs on the same platform in all cases, and other samplers are not impacted by this apparent bug.
 
- - `sklearn` v1.3.0 introduced backwards-incompatible changes in the `KMeans` sampler that changed how the random initialization affects the results, even given the same random seed. Different version of `sklearn` will affect the performance of `astartes` and we recommend including the exact version of `scikit-learn` and `astartes` used, when applicable.
-
-## Evaluate the Impact of Splitting Algorithms
-The `generate_regression_results_dict` function allows users to quickly evaluate the impact of different splitting techniques on any model supported by `sklearn`. All results are stored in a dictionary format and can be displayed in a neatly formatted table using the optional `print_results` argument.
-
-```
-from sklearn.svm import LinearSVR
-
-from astartes.utils import generate_regression_results_dict
-
-sklearn_model = LinearSVR()
-results_dict = generate_regression_results_dict(
-                    sklearn_model,
-                    X,
-                    y,
-                    print_results=True,
-               )
-
-         Train       Val      Test
-----  --------  --------  --------
-MAE   1.41522   3.13435   2.17091
-RMSE  2.03062   3.73721   2.40041
-R2    0.90745   0.80787   0.78412
-
-```
-
-## Online Documentation
-[The online documentation](https://JacksonBurns.github.io/astartes/) contains everything you see in this README with an additional tutorial for [moving from `train_test_split` in `sklearn` to `astartes`](https://jacksonburns.github.io/astartes/sklearn_to_astartes.html).
-
 ## How to Cite
 If you use `astartes` in your work please use the below citation or the "Cite this repository" button on GitHub:
 > **BibTeX**
-> @software{Burns_astartes,
->   author = {Burns, Jackson and Spiekermann, Kevin and Bhattacharjee, Himaghna and Vlachos, Dionisios and Green, William},
->   license = {MIT},
->   title = {{astartes}},
->   url = {https://github.com/JacksonBurns/astartes}
+> @software{burns_jackson_2023_8147205,
+>   author       = {Burns, Jackson and
+>                   Spiekermann, Kevin and
+>                   Bhattacharjee, Himaghna and
+>                   Vlachos, Dionisios and
+>                   Green, William},
+>   title        = {{Machine Learning Validation via Rational Dataset 
+>                    Sampling with astartes}},
+>   month        = may,
+>   year         = 2023,
+>   publisher    = {Zenodo},
+>   version      = {1.1.1},
+>   doi          = {10.5281/zenodo.8147205},
+>   url          = {https://doi.org/10.5281/zenodo.8147205}
 > }
 
-> **APA**
-> Burns, J., Spiekermann, K., Bhattacharjee, H., Vlachos, D., & Green, W. astartes [Computer software]. https://github.com/JacksonBurns/astartes
-
 ## Contributing & Developer Notes
-See [`CONTRIBUTING.md`](./CONTRIBUTING.md) for instructions on installing `asartes` for development and making a contribution.
+See [CONTRIBUTING.md](./CONTRIBUTING.md) for instructions on installing `astartes` for development, making a contribution, and general guidance on the design of `astartes`.
 
diff --git a/astartes/__init__.py b/astartes/__init__.py
index 3fad6e6a..4b790351 100644
--- a/astartes/__init__.py
+++ b/astartes/__init__.py
@@ -1,6 +1,8 @@
 # convenience import to enable 'from astartes import train_test_split'
 from .main import train_test_split, train_val_test_split
 
+__version__ = "1.1.1"
+
 # DO NOT do this:
 # from .molecules import train_test_split_molecules
 # on installations without the optional molecules install, this would cause
diff --git a/docs/index.rst b/docs/index.rst
index 2afcfd15..75ed3a60 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -11,6 +11,7 @@ astartes documentation
    :caption: Contents:
 
    README
+   CONTRIBUTING
    sklearn_to_astartes
    modules
 
diff --git a/examples/barrier_prediction_with_RDB7/RDB7_barrier_prediction_example.ipynb b/examples/barrier_prediction_with_RDB7/RDB7_barrier_prediction_example.ipynb
index 1b1cd94b..ec278d42 100644
--- a/examples/barrier_prediction_with_RDB7/RDB7_barrier_prediction_example.ipynb
+++ b/examples/barrier_prediction_with_RDB7/RDB7_barrier_prediction_example.ipynb
@@ -1,5 +1,17 @@
 {
  "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f4cad1c4",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!pip install \"astartes[demos,molecules]\"\n",
+    "!wget -N https://raw.githubusercontent.com/JacksonBurns/astartes/main/examples/barrier_prediction_with_RDB7/ccsdtf12_dz.csv"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,
@@ -42,6 +54,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "17a0b7fb",
    "metadata": {},
@@ -353,6 +366,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "471faa43",
    "metadata": {},
@@ -363,6 +377,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "92f9c10e",
    "metadata": {},
@@ -465,6 +480,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "7acf21f8",
    "metadata": {},
@@ -679,6 +695,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "6e934af9",
    "metadata": {},
diff --git a/examples/mlpds_2023_astartes_demonstration/mlpds_2023_demo.ipynb b/examples/mlpds_2023_astartes_demonstration/mlpds_2023_demo.ipynb
index 618e50fe..5bd5599f 100644
--- a/examples/mlpds_2023_astartes_demonstration/mlpds_2023_demo.ipynb
+++ b/examples/mlpds_2023_astartes_demonstration/mlpds_2023_demo.ipynb
@@ -13,6 +13,18 @@
     "Some of the constitutional isomers have been removed for simplicity in plotting."
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "2916b7b0",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!pip install \"astartes[demos,molecules]\"\n",
+    "!wget -N https://raw.githubusercontent.com/JacksonBurns/astartes/main/examples/mlpds_2023_astartes_demonstration/iupac_and_boiling.csv"
+   ]
+  },
   {
    "attachments": {},
    "cell_type": "markdown",
@@ -770,7 +782,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.3"
+   "version": "3.7.12"
   }
  },
  "nbformat": 4,
diff --git a/examples/split_comparisons/split_comparisons.ipynb b/examples/split_comparisons/split_comparisons.ipynb
index e7984f9c..513077a2 100644
--- a/examples/split_comparisons/split_comparisons.ipynb
+++ b/examples/split_comparisons/split_comparisons.ipynb
@@ -1,6 +1,7 @@
 {
  "cells": [
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "9feb8e56",
    "metadata": {},
@@ -21,10 +22,26 @@
     "Each macronutrient has a different number of calories, and by calculating a weighted sum of the grams of each, we can easily find the total calories.\n",
     "Our goal is for the model to learn this simple correlation.\n",
     "\n",
-    "The dataset used in this work was retrieved from [this link](https://www.kaggle.com/datasets/mattop/burger-king-menu-nutrition-data?resource=download) and is in the public domain (see the [CC0 license](https://creativecommons.org/share-your-work/public-domain/cc0/))."
+    "The dataset used in this work was retrieved from [this link](https://www.kaggle.com/datasets/mattop/burger-king-menu-nutrition-data?resource=download) and is in the public domain (see the [CC0 license](https://creativecommons.org/share-your-work/public-domain/cc0/)).\n",
+    "\n",
+    "Run the below cell in Google Colab to install the dependencies and retrieve the data file."
    ]
   },
   {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "436c0b85",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!pip install \"astartes[demos]\"\n",
+    "!pip install \"plotly>=4\"\n",
+    "!wget -N https://raw.githubusercontent.com/JacksonBurns/astartes/main/examples/split_comparisons/burger-king-menu.csv"
+   ]
+  },
+  {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "b3047ba2",
    "metadata": {},
@@ -35,7 +52,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": null,
    "id": "91321870",
    "metadata": {},
    "outputs": [
@@ -245,6 +262,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "4e7a876c",
    "metadata": {},
@@ -430,6 +448,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "5818321d",
    "metadata": {},
@@ -486,8 +505,6 @@
     "import plotly.express as px\n",
     "import numpy as np\n",
     "\n",
-    "plotly.offline.init_notebook_mode()\n",
-    "\n",
     "\n",
     "def scatter3d(menu, idxs=None):\n",
     "    \"\"\"Generates a nicely formatted 3D scatter plot of the Menu data\n",
@@ -558,7 +575,10 @@
     "    fig.update_layout(scene=dict(xaxis=axis_args, yaxis=axis_args, zaxis=axis_args))\n",
     "\n",
     "    # render the plot\n",
-    "    plotly.offline.iplot(fig)"
+    "    try:\n",
+    "        fig.show()\n",
+    "    except Exception as e:  # workaround to allow testing in GitHub actions\n",
+    "        print(\"Could not .show() plot, are you running in GitHub actions?\", str(e))"
    ]
   },
   {
@@ -1863,6 +1883,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "1017bb28",
    "metadata": {},
@@ -1873,6 +1894,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "f6a36594",
    "metadata": {},
@@ -1904,6 +1926,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "3e47fefc",
    "metadata": {},
@@ -1957,6 +1980,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "57fef880",
    "metadata": {},
@@ -3417,6 +3441,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "7f2157a6",
    "metadata": {},
@@ -3429,6 +3454,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "479b5494",
    "metadata": {},
@@ -4937,6 +4963,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "8ffb431b",
    "metadata": {},
@@ -4949,6 +4976,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "5c949327",
    "metadata": {},
@@ -6452,6 +6480,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "9a61a619",
    "metadata": {},
@@ -6460,6 +6489,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "c339d197",
    "metadata": {},
@@ -6482,6 +6512,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "0437922c",
    "metadata": {},
@@ -6561,6 +6592,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "510e2a56",
    "metadata": {},
@@ -8022,6 +8054,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "0a4e6e11",
    "metadata": {},
@@ -8068,6 +8101,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "57163507",
    "metadata": {},
@@ -8080,6 +8114,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "059d028f",
    "metadata": {},
@@ -9594,6 +9629,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "ac781fa0",
    "metadata": {},
@@ -9628,6 +9664,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "af65b1e2",
    "metadata": {},
@@ -9637,6 +9674,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "90772dc5",
    "metadata": {},
@@ -11151,6 +11189,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "6fa20991",
    "metadata": {},
@@ -11160,6 +11199,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "7b575c5a",
    "metadata": {},
@@ -11178,6 +11218,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "92c66f0e",
    "metadata": {},
@@ -11224,6 +11265,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "0306650f",
    "metadata": {},
@@ -11247,6 +11289,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "2579de49",
    "metadata": {},
@@ -11320,6 +11363,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "ef10fc44",
    "metadata": {},
@@ -11335,6 +11379,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "f3a65785",
    "metadata": {},
@@ -11388,6 +11433,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "8bec3707",
    "metadata": {},
@@ -11473,6 +11519,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "37563c3a",
    "metadata": {},
@@ -11524,6 +11571,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "6acd224c",
    "metadata": {},
@@ -11556,6 +11604,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "ac41f434",
    "metadata": {},
@@ -11565,6 +11614,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "df4a3955",
    "metadata": {},
diff --git a/pyproject.toml b/pyproject.toml
index 82b9d317..3e3888ce 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,6 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "astartes"
-version = "1.1.1"
 authors = [
     { name = "Jackson Burns", email = "jwburns@mit.edu" },
     { name = "Himaghna Bhattacharjee", email = "himaghna@udel.edu" },
@@ -22,6 +21,10 @@ classifiers = [
 urls = { Homepage = "https://github.com/JacksonBurns/astartes" }
 requires-python = ">=3.7"
 dependencies = ["scikit_learn", "tabulate", "numpy", "scipy", "pandas"]
+dynamic = ["version"]
+
+[tool.setuptools.dynamic]
+version = { attr = "astartes.__version__" }
 
 [project.optional-dependencies]
 molecules = ["aimsim"]


Test patch:
diff --git a/examples/train_val_test_split_sklearn_example/train_val_test_split_example.ipynb b/examples/train_val_test_split_sklearn_example/train_val_test_split_example.ipynb
index 8126ecd1..c3c03682 100644
--- a/examples/train_val_test_split_sklearn_example/train_val_test_split_example.ipynb
+++ b/examples/train_val_test_split_sklearn_example/train_val_test_split_example.ipynb
@@ -1,5 +1,16 @@
 {
  "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "6436f30b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%%capture\n",
+    "!pip install \"astartes[demos]\""
+   ]
+  },
   {
    "attachments": {},
    "cell_type": "markdown",
@@ -118,6 +129,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "bb00d17a",
    "metadata": {},
@@ -171,6 +183,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "5fc16d62",
    "metadata": {},
@@ -202,6 +215,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "6ff82d2e",
    "metadata": {},
@@ -234,6 +248,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "7d9f5f88",
    "metadata": {},
@@ -242,6 +257,7 @@
    ]
   },
   {
+   "attachments": {},
    "cell_type": "markdown",
    "id": "0cb5bf22",
    "metadata": {},