Repository: RecList/reclist (language: python)

Issue:
Develop - adding pre-commit 
@patrickjohncyh  @jacopotagliabue   Adding pre-commit hooks 


Discussion / hints:

Fix patch:
diff --git a/.gitignore b/.gitignore
index 17476b0..328ccaf 100644
--- a/.gitignore
+++ b/.gitignore
@@ -112,3 +112,9 @@ ENV/
 
 # mac file
 .DS_Store
+
+*.png
+
+.neptune/
+
+evalrs_dataset_KDD23/
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
new file mode 100644
index 0000000..7ccd391
--- /dev/null
+++ b/.pre-commit-config.yaml
@@ -0,0 +1,22 @@
+# See https://pre-commit.com for more information
+# See https://pre-commit.com/hooks.html for more hooks
+repos:
+-   repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v3.2.0
+    hooks:
+    -   id: trailing-whitespace
+    -   id: end-of-file-fixer
+    -   id: check-yaml
+    -   id: check-added-large-files
+
+-   repo: https://github.com/psf/black
+    rev: 22.3.0
+    hooks:
+    -   id: black
+
+-   repo: https://github.com/pycqa/isort
+    rev: 5.10.1
+    hooks:
+      - id: isort
+        name: isort (python)
+        args: ["--profile", "black", "--filter-files"]
diff --git a/LICENSE b/LICENSE
index d41cc61..d90f872 100644
--- a/LICENSE
+++ b/LICENSE
@@ -19,4 +19,3 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 SOFTWARE.
-
diff --git a/README.rst b/README.rst
index 1dc1789..8869dbd 100644
--- a/README.rst
+++ b/README.rst
@@ -2,6 +2,10 @@
 RecList
 =======
 
+.. image:: images/reclist.png
+        :width: 30%
+        :alt: Rocket Emoji
+
 
 .. image:: https://img.shields.io/pypi/v/reclist.svg
         :target: https://pypi.python.org/pypi/reclist
@@ -30,11 +34,6 @@ RecList
         :alt: YouTube
 
 
-RecList
-
-
-* Free software: MIT license
-* Documentation: https://reclist.readthedocs.io.
 
 Overview
 --------
@@ -46,57 +45,40 @@ While quantitative metrics over held-out data points are important, a lot more t
 to properly function in the wild and not erode our confidence in them: for example, a model may boast an accuracy improvement over the entire dataset, but actually be significantly worse than another on rare items or new users; or again, a model that correctly recommends HDMI cables as add-on for shoppers buying a TV, may also wrongly  recommend TVs to shoppers just buying a cable.
 
 *RecList* goal is to operationalize these important intuitions into a practical package for testing research and production models in a more nuanced way, without
-requiring unnecessary custom code and ad hoc procedures. To streamline comparisons among existing models, *RecList* ships with popular datasets and ready-made behavioral tests: read the `TDS blog post <https://towardsdatascience.com/ndcg-is-not-all-you-need-24eb6d2f1227>`__ as a gentle introduction to the main use cases, check the `paper <https://dl.acm.org/doi/abs/10.1145/3487553.3524215>`__ for more details on the relevant literature.
-
-If you are not familiar with the library, we suggest first taking our small tour to get acquainted with the main abstractions through ready-made models and public datasets.
-
-Quick Links
-~~~~~~~~~~~
-
-* Our `talk <https://www.youtube.com/watch?v=cAlJYxFYA04>`__ at Stitch Fix Algo Hour, for a general presentation.
-* Our original `paper <https://dl.acm.org/doi/abs/10.1145/3487553.3524215>`__, with in-depth analysis, detailed use cases and scholarly references (`arxiv <https://arxiv.org/abs/2111.09963>`__).
-* Our CIKM Data Challenge (`paper <https://rdcu.be/c4dXy>`__, `code <https://github.com/RecList/evalRS-CIKM-2022>`__) for music recommendations, based on RecList.
-* A `colab notebook <https://colab.research.google.com/drive/1Wn5mm0csEkyWqmBBDxNBkfGR6CNfWeH-?usp=sharing>`__, showing how to train a cart recommender model from scratch and use the library to test it.
-* Our `blog post <https://towardsdatascience.com/ndcg-is-not-all-you-need-24eb6d2f1227>`__, with examples and practical tips.
-
-
-Supporters
-~~~~~~~~~~
-RecList is a community project made possible by the generous support of awesome folks. Between June and December 2022, RecList has been supported by:
+requiring unnecessary custom code and ad hoc procedures.
 
-Comet
-=====
+If you are not familiar with the library, we suggest first taking our small tour to get acquainted with the main abstractions through ready-made models and tests.
 
-.. image:: https://github.com/jacopotagliabue/reclist/raw/main/images/comet.png
-   :target: https://www.comet.com/?utm_source=jacopot&utm_medium=referral&utm_campaign=online_jacopot_2022&utm_content=github_reclist
-   :width: 100
+Starting
+~~~~~~~~
 
-Neptune
-=======
 
-.. image:: https://github.com/jacopotagliabue/reclist/raw/main/images/neptune.png
-   :target: https://neptune.ai
-   :width: 100
 
-Gantry
-======
+.. |colab1_tutorial| image:: https://colab.research.google.com/assets/colab-badge.svg
+    :target: https://colab.research.google.com/drive/1GVsVB1a3H9qbRQvwtb0TBDxq8A5nXc5w?usp=sharing
+    :alt: Open In Colab
 
-.. image:: https://github.com/jacopotagliabue/reclist/raw/main/images/gantry.png
-   :target: https://gantry.io/
-   :width: 100
 
++--------------------------------------------------------------------------------+------------------+
+| Name                                                                           | Link             |
++================================================================================+==================+
+| Tutorial 101 - Introduction to Reclist                                         | |colab1_tutorial||
++--------------------------------------------------------------------------------+------------------+
 
-Project updates
-~~~~~~~~~~~~~~~
+Quick Links
+~~~~~~~~~~~
 
-*Community Support*: RecList is an open source community project made possible by the support of the awesome folks at `Comet <https://www.comet.ml/site/>`__, `Neptune <https://neptune.ai/>`__ and `Gantry <https://gantry.io/>`__.
-Soon RecList tests will be natively integrated with the MLOps tools you already know and love!
+* The original `paper <https://dl.acm.org/doi/abs/10.1145/3487553.3524215>`__ (`arxiv <https://arxiv.org/abs/2111.09963>`__) and initial `release post <https://towardsdatascience.com/ndcg-is-not-all-you-need-24eb6d2f1227>`__.
+* `EvalRS22@CIKM <https://github.com/RecList/evalRS-CIKM-2022>`__ and `EvalRS23@KDD <https://reclist.io/kdd2023-cup/>`__ , for music recommendations with RecList.
+* A `colab notebook <https://colab.research.google.com/drive/1GVsVB1a3H9qbRQvwtb0TBDxq8A5nXc5w>`__, for a quick interactive tour.
+* Our `website <https://reclist.io/>`__ for past talks and presentations.
 
-*January 2023*: We concluded the RecList-powered `Data Challenge <https://reclist.io/cikm2022-cup/>`__ at CIKM 2022 and released all the materials to the community. We shared our learnings in a `paper <https://www.nature.com/articles/s42256-022-00606-0>`__ on Nature Machine Intelligence, and prepare the release of RecList Beta.
 
-In the last few months, we presented this library to practioners at Tubi, eBay, NVIDIA, BBC and other RecSys companies: we are in the process of collecting our thoughts after all the feedback we received, as we plan a beta release for this package in the next few months - come back often for updates, as we will also open a call for collaboration!
+Status
+~~~~~~~~~~~
 
-Please remember that the library is in alpha (i.e. enough working code to finish the paper and tinker with it). We welcome early feedback, but please be advised that the package may change substantially in the near future ("If you're not embarrassed by the first version, you've launched too late").
+* *RecList* is free software released under the MIT license, and it has been adopted by popular `open-source <https://github.com/RecList/evalRS-CIKM-2022>`__  `data challenges <https://reclist.io/kdd2023-cup/>`__.
+* After a major API re-factoring, *RecList* is now in *beta*.
 
 Summary
 ~~~~~~~
@@ -106,17 +88,13 @@ This doc is structured as follows:
 * `Quick Start`_
 * `A Guided Tour`_
 * `Capabilities`_
-* `Roadmap`_
-* `Talks and Presentations`_
 * `License and Citation`_
 * `Acknowledgments`_
 
 Quick Start
 -----------
 
-If you want to see *RecList* in action, clone the repository, create and activate a virtual env, and install the required packages from pip (you can install from root of course). If you prefer to experiment in an interactive, no-installation-required fashion, try out our `colab notebook <https://colab.research.google.com/drive/1Wn5mm0csEkyWqmBBDxNBkfGR6CNfWeH-?usp=sharing>`__.
-
-Sample scripts are divided by use-cases: similar items, complementary items or session-based recommendations. When executing one, a suitable public dataset will be downloaded, and a baseline model trained: finally, the script will run a pre-made suite of behavioral tests to show typical results.
+If you want to see *RecList* in action, clone the repository, create and activate a virtual env, and install the required packages from pip (you can install from root of course). If you prefer to experiment in an interactive, no-installation-required fashion, try out our `colab notebook <https://colab.research.google.com/drive/1GVsVB1a3H9qbRQvwtb0TBDxq8A5nXc5w>`__.
 
 .. code-block:: bash
 
@@ -125,84 +103,78 @@ Sample scripts are divided by use-cases: similar items, complementary items or s
     python3 -m venv venv
     source venv/bin/activate
     pip install reclist
-    python examples/coveo_complementary_rec.py
-
-Running *your* model on one of the supported dataset, leveraging the pre-made tests, is as easy as implementing a simple interface, *RecModel*.
+    python examples/evalrs_2023.py
 
 Once you've run successfully the sample script, take the guided tour below to learn more about the abstractions and the out-of-the-box capabilities of *RecList*.
 
 A Guided Tour
 -------------
 
-An instance of `RecList <https://github.com/jacopotagliabue/reclist/blob/main/reclist/reclist.py>`__ represents a suite of tests for recommender systems: given
-a dataset (more appropriately, an instance of `RecDataset <https://github.com/jacopotagliabue/reclist/blob/main/reclist/abstractions.py>`__)
-and a model (an instance of `RecModel <https://github.com/jacopotagliabue/reclist/blob/main/reclist/abstractions.py>`__), it will run the specified tests on the target dataset, using the supplied model.
+An instance of `RecList <https://github.com/jacopotagliabue/reclist/blob/main/reclist/reclist.py>`__ represents a suite of tests for recommender systems.
+
+As the sample `examples/evalrs_2023.py` script shows, we leave users quite a large range of options: we provide out of the box standard metrics
+in case your dataset is DataFrame-shaped (or you can / wish turn it into such a shape), but don't force you any pattern if you just want to use *RecList*
+for the scaffolding it provides.
 
-For example, the following code instantiates a pre-made suite of tests that contains sensible defaults for a `cart recommendation use case <https://github.com/jacopotagliabue/reclist/blob/main/reclist/reclist.py>`__:
+For example, the following code only assumes you have a dataset with golden labels, predictions, and metadata (e.g. item features) in the form of a DataFrame:
 
 .. code-block:: python
 
-    rec_list = CoveoCartRecList(
-        model=model,
-        dataset=coveo_dataset
+    cdf = DFSessionRecList(
+        dataset=df_events,
+        model_name="myDataFrameRandomModel",
+        predictions=df_predictions,
+        y_test=df_dataset,
+        logger=LOGGER.NEPTUNE,
+        metadata_store= METADATA_STORE.LOCAL,
+        similarity_model=my_sim_model,
+        bucket=os.environ["S3_BUCKET"],
+        NEPTUNE_KEY=os.environ["NEPTUNE_KEY"],
+        NEPTUNE_PROJECT_NAME=os.environ["NEPTUNE_PROJECT_NAME"],
     )
-    # invoke rec_list to run tests
-    rec_list(verbose=True)
 
-Our library pre-packages standard recSys KPIs and important behavioral tests, divided by use cases, but it is built with extensibility in mind: you can re-use tests in new suites, or you can write new domain-specific suites and tests.
+    # run reclist
+    cdf(verbose=True)
 
-Any suite must inherit the *RecList* interface, and then declare with Pytonic decorators its tests. In this case, the test re-uses a standard function:
+Our library pre-packages standard recSys KPIs and important behavioral tests, but it is built with extensibility in mind: you can re-use tests in new suites, or you can write new domain-specific suites and tests.
+Any suite must inherit from the main interface, and then declare its tests as functions decorated with *@rec_test*. In the example, an instance is created with one slice-based test: the decorator and return type are used to automatically generate a chart.
 
 .. code-block:: python
 
     class MyRecList(RecList):
 
-        @rec_test(test_type='stats')
-        def basic_stats(self):
+        @rec_test(test_type="AccuracyByCountry", display_type=CHART_TYPE.BARS)
+        def accuracy_by_country(self):
             """
-            Basic statistics on training, test and prediction data
+            Compute the accuracy by country
+
+            NOTE: the accuracy here is just a random number.
             """
-            from reclist.metrics.standard_metrics import statistics
-            return statistics(self._x_train,
-                self._y_train,
-                self._x_test,
-                self._y_test,
-                self._y_preds)
+            from random import randint
+            return {"US": randint(0, 100), "CA": randint(0, 100), "FR": randint(0, 100) }
 
 
-Any model can be tested, as long as its predictions are wrapped in a *RecModel*. This allows for pure "black-box" testings,
-a SaaS provider can be tested just by wrapping the proper API call in the method:
+Inheritance is powerful, as we can build new suites by re-using existing ones. Here, we inherit the tests from an existing "parent" list and just add one more to create a new suite:
 
 .. code-block:: python
 
-    class MyCartModel(RecModel):
-
-        def __init__(self, **kwargs):
-            super().__init__(**kwargs)
+    class ChildRecList(MyParentRecList):
 
-        def predict(self, prediction_input: list, *args, **kwargs):
+        @rec_test(test_type='custom_test', display_type=CHART_TYPE.SCALAR)
+        def my_test(self):
             """
-            Implement the abstract method, accepting a list of lists, each list being
-            the content of a cart: the predictions returned by the model are the top K
-            items suggested to complete the cart.
+            Custom test, returning my lucky number as an example
             """
+            from random import randint
 
-            return
+            return { "luck_number": randint(0, 100) }
 
-More generally, the logical workflow of a typical RecList implementation is as follows (see our `blog post <https://towardsdatascience.com/ndcg-is-not-all-you-need-24eb6d2f1227>`__ for a longer explanation):
 
-.. image:: https://github.com/jacopotagliabue/reclist/blob/main/images/workflow.gif
-   :height: 400
+Any model can be tested, as no assumption is made on the model's structure, but only the availability of *predictions*
+and *ground truth*. Once again, while our example leverages a DataFrame-shaped dataset for these entities, you are free to build your own
+RecList instance with any shape you prefer, provided you implement the metrics accordingly.
 
-While many standard KPIs are available in the package, the philosophy behind *RecList* is that metrics like Hit Rate provide only a partial picture
-of the expected behavior of recommenders in the wild: two models with very similar accuracy can have very different behavior on, say, the long-tail, or
-model A can be better than model B overall, but at the expense of providing disastrous performance on a set of inputs that are particularly important in production.
-
-*RecList* recognizes that outside of academic benchmarks, some mistakes are worse than others, and not all inputs are created equal: when possible, it tries
-to operationalize through scalable code behavioral insights for debugging and error analysis; it also
-provides extensible abstractions when domain knowledge and custom logic are needed.
-
-Once you run a suite of tests, results are dumped automatically and versioned in a local folder, structured as follows
+Once you run a suite of tests, results are dumped automatically and versioned in a folder (local or on S3), structured as follows
 (name of the suite, name of the model, run timestamp):
 
 .. code-block::
@@ -213,78 +185,25 @@ Once you run a suite of tests, results are dumped automatically and versioned in
           1637357392/
           1637357404/
 
-If you start using *RecList* as part of your standard testings - either for research or production purposes - you can use the JSON report
-for machine-to-machine communication with downstream system (e.g. you may want to automatically fail the model pipeline if certain behavioral tests are not passed).
-
-Note: our app is deprecated, as RecList Beta will have connectors with existing apps (experiment trackers, model cards, etc.).
+If you use *RecList* as part of your standard testings - either for research or production purposes - you can use the JSON report
+for machine-to-machine communication with downstream system (e.g. you may want to automatically fail the `pipeline <https://github.com/jacopotagliabue/recs-at-resonable-scale>`__  if tests are not passed).
 
 Capabilities
 ------------
 
-*RecList* provides a dataset and model agnostic framework to scale up behavioral tests. As long as the proper abstractions
-are implemented, all the out-of-the-box components can be re-used. For example:
-
-* you can use a public dataset provided by *RecList* to train your new cart recommender model, and then use the *RecTests* we provide for that use case;
-
-* you can use some baseline model on your custom dataset, to establish a baseline for your project;
-
-* you can use a custom model, on a private dataset and define from scratch a new suite of tests, mixing existing methods and domain-specific tests.
-
-We list below what we currently support out-of-the-box, with particular focus on datasets and tests, as the models we provide
-are convenient baselines, but they are not meant to be SOTA research models.
-
-Datasets
-~~~~~~~~
-
-RecList features convenient wrappers around popular datasets, to help test models over known benchmarks
-in a standardized way.
+*RecList* provides a dataset and model agnostic framework to scale up behavioral tests. We provide some suggested abstractions
+based on DataFrames to make existing tests and metrics fully re-usable, but we don't force any pattern on the user. As out of the box functionality, the package provides:
 
-* `Coveo Data Challenge <https://github.com/coveooss/SIGIR-ecom-data-challenge>`__
-* (a smaller version of) `The Million Playlist Dataset <https://engineering.atspotify.com/2018/05/30/introducing-the-million-playlist-dataset-and-recsys-challenge-2018/>`__
-* (a smaller version of) `MovieLens <https://grouplens.org/datasets/movielens/>`__
+* tests and metrics to be used on your own datasets and models;
 
-Behavioral Tests
-~~~~~~~~~~~~~~~~
+* automated storage of results, with versioning, both in a local folder or on S3;
 
-RecList helps report standard quantitative metrics over popular (or custom) datasets, such as the ones collected in
-*standard_metrics.py*: hit rate, mrr, coverage, popularity bias, etc. However, RecList raison d'etre is providing plug-and-play
-behavioral tests, as agnostic as possible to the underlying models and datasets, while leaving open the possibility of writing
-personalized tests when domain knowledge and custom logic are necessary.
+* flexible, Python interface to declare tests-as-functions, and annotate them with *display_type* for automated charts;
 
-Tests descriptions are available in our (WIP) `docs <https://reclist.readthedocs.io>`__, but we share here some examples from our `paper <https://dl.acm.org/doi/abs/10.1145/3487553.3524215>`__.
+* pre-built connectors with popular experiment trackers (e.g. Neptune, Comet), and an extensible interface to add your own;
 
-First, RecList allows to compare the performance of models which may have similar aggregate KPIs (e.g. hit rate on the entire
-test set) in different slices. When plotting HR by product popularity, it is easy to spot that
-prod2vec works much better with rarer items than the alternatives:
+* reference implementations based on popular data challenges that used RecList.
 
-.. image:: https://github.com/jacopotagliabue/reclist/blob/main/images/hit_rate_dist.png
-   :height: 175
-
-When slicing by important meta-data (in this simulated example, brands), RecList uncovers significant differences
-in performance for different groups; since the features we care about vary across datasets,
-the package allows for a generic way to partition the test set and compute per-slice metrics:
-
-.. image:: https://github.com/jacopotagliabue/reclist/blob/main/images/slice_dist.png
-   :height: 175
-
-Finally, RecList can take advantage of the latent item space to compute the cosine distances <query item, ground truth> and
-<query item, prediction> for missed predictions in the test set. In a cart recommender use case, we expect items to
-reflect the complementary nature of the suggestions: if a TV is in the cart, a model should recommend a HDMI cable,
-not another TV. As we see in the comparison below, Google's predictions better match the label distribution,
-suggesting that the model better capture the nature of the task:
-
-.. image:: https://github.com/jacopotagliabue/reclist/blob/main/images/distance_to_query.png
-   :height: 175
-
-Roadmap
--------
-
-We have exciting news about our Beta, including the usage of RecList as main library for the `CIKM Data Challenge <https://reclist.io/cikm2022-cup/>`__!
-
-Contributing
-~~~~~~~~~~~~
-
-We will update this repo with some guidelines for contributions as soon as the codebase becomes more stable. Check back often for updates!
 
 Acknowledgments
 ---------------
@@ -297,12 +216,13 @@ The original authors are:
 * Chloe He - `LinkedIn <https://www.linkedin.com/in/chloe-he//>`__, `GitHub <https://github.com/chloeh13q>`__
 * Brian Ko - `LinkedIn <https://www.linkedin.com/in/briankosw/>`__, `GitHub <https://github.com/briankosw>`__
 
-If you have questions or feedback, please reach out to: :code:`jacopo dot tagliabue at tooso dot ai`.
+*RecList* is a community project made possible by the generous support of awesome folks. Between June and December 2022, the development of our beta has been supported by `Comet <https://www.comet.com/>`__, `Neptune <https://neptune.ai/homepage>`__ , `Gantry <https://gantry.io/>`__.
+Our beta has been developed with the help of:
+
+* Unnati Patel - `LinkedIn <https://www.linkedin.com/in/unnati-p-16626610a/>`__
 
-Talks and Presentations
------------------------
 
-Past and upcoming talks and presentations can be found at our new `website <https://reclist.io/>`__.
+If you have questions or feedback, please reach out to: :code:`jacopo dot tagliabue at nyu dot edu`.
 
 License and Citation
 --------------------
diff --git a/docs/conf.py b/docs/conf.py
index fdb4a48..d63fdab 100755
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -19,7 +19,8 @@
 #
 import os
 import sys
-sys.path.insert(0, os.path.abspath('..'))
+
+sys.path.insert(0, os.path.abspath(".."))
 
 import reclist
 
@@ -31,22 +32,22 @@
 
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
-extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode']
+extensions = ["sphinx.ext.autodoc", "sphinx.ext.viewcode"]
 
 # Add any paths that contain templates here, relative to this directory.
-templates_path = ['_templates']
+templates_path = ["_templates"]
 
 # The suffix(es) of source filenames.
 # You can specify multiple suffix as a list of string:
 #
 # source_suffix = ['.rst', '.md']
-source_suffix = '.rst'
+source_suffix = ".rst"
 
 # The master toctree document.
-master_doc = 'index'
+master_doc = "index"
 
 # General information about the project.
-project = 'RecList'
+project = "RecList"
 copyright = "2021, Federico Bianchi"
 author = "Federico Bianchi"
 
@@ -69,10 +70,10 @@
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This patterns also effect to html_static_path and html_extra_path
-exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
+exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
 
 # The name of the Pygments (syntax highlighting) style to use.
-pygments_style = 'sphinx'
+pygments_style = "sphinx"
 
 # If true, `todo` and `todoList` produce output, else they produce nothing.
 todo_include_todos = False
@@ -83,7 +84,7 @@
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-html_theme = 'sphinx_rtd_theme'
+html_theme = "sphinx_rtd_theme"
 
 # Theme options are theme-specific and customize the look and feel of a
 # theme further.  For a list of options available for each theme, see the
@@ -94,13 +95,13 @@
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
-html_static_path = ['_static']
+html_static_path = ["_static"]
 
 
 # -- Options for HTMLHelp output ---------------------------------------
 
 # Output file base name for HTML help builder.
-htmlhelp_basename = 'reclistdoc'
+htmlhelp_basename = "reclistdoc"
 
 
 # -- Options for LaTeX output ------------------------------------------
@@ -109,15 +110,12 @@
     # The paper size ('letterpaper' or 'a4paper').
     #
     # 'papersize': 'letterpaper',
-
     # The font size ('10pt', '11pt' or '12pt').
     #
     # 'pointsize': '10pt',
-
     # Additional stuff for the LaTeX preamble.
     #
     # 'preamble': '',
-
     # Latex figure (float) alignment
     #
     # 'figure_align': 'htbp',
@@ -127,9 +125,7 @@
 # (source start file, target name, title, author, documentclass
 # [howto, manual, or own class]).
 latex_documents = [
-    (master_doc, 'reclist.tex',
-     'RecList Documentation',
-     'Federico Bianchi', 'manual'),
+    (master_doc, "reclist.tex", "RecList Documentation", "Federico Bianchi", "manual"),
 ]
 
 
@@ -137,11 +133,7 @@
 
 # One entry per manual page. List of tuples
 # (source start file, name, description, authors, manual section).
-man_pages = [
-    (master_doc, 'reclist',
-     'RecList Documentation',
-     [author], 1)
-]
+man_pages = [(master_doc, "reclist", "RecList Documentation", [author], 1)]
 
 
 # -- Options for Texinfo output ----------------------------------------
@@ -150,13 +142,13 @@
 # (source start file, target name, title, author,
 #  dir menu entry, description, category)
 texinfo_documents = [
-    (master_doc, 'reclist',
-     'RecList Documentation',
-     author,
-     'reclist',
-     'One line description of project.',
-     'Miscellaneous'),
+    (
+        master_doc,
+        "reclist",
+        "RecList Documentation",
+        author,
+        "reclist",
+        "One line description of project.",
+        "Miscellaneous",
+    ),
 ]
-
-
-
diff --git a/examples/coveo_complementary_rec.py b/examples/coveo_complementary_rec.py
deleted file mode 100644
index c8c5b73..0000000
--- a/examples/coveo_complementary_rec.py
+++ /dev/null
@@ -1,36 +0,0 @@
-"""
-    This is a sample script showing how to get started with reclist. In particular,
-    we are targeting a "complementary items" use cases, such as for example a cart recommender:
-    if a shopper added item X to the cart, what is she likely to add next?
-
-    We train a simple, yet effective prod2vec baseline model (https://arxiv.org/abs/2007.14906),
-    re-using for convenience a "training embedding" function already implemented by recsys. The script
-    show how easy it is to run behavioral tests on a target dataset, in this case a wrapper around a large
-    e-commerce dataset (the Coveo Data Challenge dataset: https://github.com/coveooss/SIGIR-ecom-data-challenge).
-
-    If you want to run your own model on the same dataset and the same behavioral test suite, just implement
-    your model prediction using the RecModel, and pass it to the CartList.
-"""
-from reclist.datasets import CoveoDataset
-from reclist.recommenders.prod2vec import CoveoP2VRecModel
-from reclist.reclist import CoveoCartRecList
-
-if __name__ == "__main__":
-
-    # get the coveo data challenge dataset as a RecDataset object
-    coveo_dataset = CoveoDataset()
-
-    # re-use a skip-gram model from reclist to train a latent product space, to be used
-    # (through knn) to build a recommender
-    model = CoveoP2VRecModel()
-    model.train(coveo_dataset.x_train)
-
-    # instantiate rec_list object, prepared with standard quantitative tests
-    # and sensible behavioral tests (check the paper for details!)
-    rec_list = CoveoCartRecList(
-        model=model,
-        dataset=coveo_dataset
-    )
-    # invoke rec_list to run tests
-    rec_list(verbose=True)
-
diff --git a/examples/dummy.py b/examples/dummy.py
new file mode 100644
index 0000000..aba30cb
--- /dev/null
+++ b/examples/dummy.py
@@ -0,0 +1,125 @@
+import numpy as np
+from reclist.logs import LOGGER
+from reclist.similarity_models import FakeSimilarityModel
+from reclist.metadata import METADATA_STORE
+from reclist.reclist import RecList
+from random import randint, choice
+import os
+from reclist.reclist import rec_test
+from reclist.charts import CHART_TYPE
+
+class FreeSessionRecList(RecList):
+
+    def __init__(
+        self,
+        dataset,
+        metadata,
+        predictions,
+        model_name,
+        logger: LOGGER,
+        metadata_store: METADATA_STORE,
+        **kwargs
+    ):
+        super().__init__(
+            model_name,
+            logger,
+            metadata_store,
+            **kwargs
+        )
+        self.dataset = dataset
+        self.metadata = metadata
+        self.predictions = predictions
+        self.similarity_model = kwargs.get("similarity_model", None)
+
+        return
+
+    @rec_test(test_type="LessWrong", display_type=CHART_TYPE.SCALAR)
+    def less_wrong(self):
+        truths = self.dataset
+        predictions = self.predictions
+        model_misses = [(t, p) for t, p in zip(truths, predictions) if t != p]
+        similarity_scores = [
+            self.similarity_model.similarity_gradient(t, p) for t, p in model_misses
+        ]
+
+        return np.average(similarity_scores)
+
+    @rec_test(test_type="SlicedAccuracy", display_type=CHART_TYPE.SCALAR)
+    def sliced_accuracy(self):
+        """
+        Compute the accuracy by slice
+        """
+        from reclist.metrics.standard_metrics import accuracy_per_slice
+
+        return accuracy_per_slice(
+            self.dataset, self.predictions, self.metadata["categories"]
+        )
+
+    @rec_test(test_type="Accuracy", display_type=CHART_TYPE.SCALAR)
+    def accuracy(self):
+        """
+        Compute the accuracy
+        """
+        from sklearn.metrics import accuracy_score
+
+        return accuracy_score(self.dataset, self.predictions)
+
+    @rec_test(test_type="AccuracyByCountry", display_type=CHART_TYPE.BARS)
+    def accuracy_by_country(self):
+        """
+        Compute the accuracy by country
+        """
+        # TODO: note that is a static test, used to showcase the bin display
+        from random import randint
+        return {"US": randint(0, 100), "CA": randint(0, 100), "FR": randint(0, 100) }
+
+
+try:
+    from dotenv import load_dotenv
+    load_dotenv()
+except:
+    print("Dotenv not loaded: if you need ENV variables, make sure you export them manually")
+
+
+class DummyModel:
+
+    def __init__(self, n):
+        self.n = n
+
+    def predict(self):
+        """
+        Do something
+        """
+        from random import randint
+        return [randint(0, 1) for _ in range(self.n)]
+
+
+# create a dataset randomly
+
+n = 10000
+apo_model = DummyModel(n)
+dataset = [randint(0, 1) for _ in range(n)]
+metadata = {"categories": [choice(["cat", "dog", "capybara"]) for _ in range(n)]}
+predictions = apo_model.predict()
+my_sim_model = FakeSimilarityModel()
+assert len(dataset) == len(predictions), "dataset, predictions must have the same length"
+
+
+# initialize with everything
+cd = FreeSessionRecList(
+    dataset=dataset,
+    metadata=metadata,
+    predictions=predictions,
+    model_name="myRandomModel",
+    logger=LOGGER.COMET,
+    metadata_store= METADATA_STORE.LOCAL,
+    similarity_model=my_sim_model,
+    bucket=os.environ["S3_BUCKET"],
+    COMET_KEY=os.environ["COMET_KEY"],
+    COMET_PROJECT_NAME=os.environ["COMET_PROJECT_NAME"],
+    COMET_WORKSPACE=os.environ["COMET_WORKSPACE"],
+)
+
+# run reclist
+cd(verbose=True)
+
diff --git a/examples/evalrs_2023.py b/examples/evalrs_2023.py
new file mode 100644
index 0000000..c34653b
--- /dev/null
+++ b/examples/evalrs_2023.py
@@ -0,0 +1,140 @@
+from reclist.logs import LOGGER
+from reclist.metadata import METADATA_STORE
+import pandas as pd
+import numpy as np
+import os
+from reclist.reclist import rec_test
+from reclist.reclist import RecList
+from random import choice
+
+class DFSessionRecList(RecList):
+
+    def __init__(
+        self,
+        dataset,
+        predictions,
+        model_name,
+        logger: LOGGER,
+        metadata_store: METADATA_STORE,
+        **kwargs
+    ):
+        super().__init__(
+            model_name,
+            logger,
+            metadata_store,
+            **kwargs
+        )
+        self.dataset = dataset
+        self._y_preds = predictions
+        self._y_test = kwargs.get("y_test", None)
+        self.similarity_model = kwargs.get("similarity_model", None)
+
+        return
+
+    @rec_test('HIT_RATE')
+    def hit_rate_at_100(self):
+        hr = self.hit_rate_at_k(self._y_preds, self._y_test, k=100)
+        return hr
+
+    def hit_rate_at_k(self, y_pred: pd.DataFrame, y_test: pd.DataFrame, k: int):
+        """
+        N = number test cases
+        M = number ground truth per test case
+        """
+        hits = self.hits_at_k(y_pred, y_test, k)  # N x M x k
+        hits = hits.max(axis=1)  # N x k
+        return hits.max(axis=1).mean()  # 1
+
+    def hits_at_k(self, y_pred: pd.DataFrame, y_test: pd.DataFrame, k: int):
+        """
+        N = number test cases
+        M = number ground truth per test case
+        """
+        y_test_mask = y_test.values != -1  # N x M
+
+        y_pred_mask = y_pred.values[:, :k] != -1  # N x k
+
+        y_test = y_test.values[:, :, None]  # N x M x 1
+        y_pred = y_pred.values[:, None, :k]  # N x 1 x k
+
+        hits = y_test == y_pred  # N x M x k
+        hits = hits * y_test_mask[:, :, None]  # N x M x k
+        hits = hits * y_pred_mask[:, None, :]  # N x M x k
+
+        return hits
+
+
+class EvalRSSimpleModel:
+    """
+    This is a dummy model that returns random predictions on the EvalRS dataset.
+    """
+    def __init__(self, items: pd.DataFrame, top_k: int=10, **kwargs):
+        self.items = items
+        self.top_k = top_k
+        print("Received additional arguments: {}".format(kwargs))
+        return
+
+    def predict(self, user_ids: pd.DataFrame) -> pd.DataFrame:
+        k = self.top_k
+        num_users = len(user_ids)
+        pred = self.items.sample(n=k*num_users, replace=True).index.values
+        pred = pred.reshape(num_users, k)
+        pred = np.concatenate((user_ids[['user_id']].values, pred), axis=1)
+        return pd.DataFrame(pred, columns=['user_id', *[str(i) for i in range(k)]]).set_index('user_id')
+
+
+"""
+    We now load the dataset
+"""
+
+print("\n\n ======> Loading dataset. \n\n")
+df_events = pd.read_csv('evalrs_dataset_KDD23/evalrs_events.csv', index_col=0, dtype='int32')
+df_tracks = pd.read_csv('evalrs_dataset_KDD23/evalrs_tracks.csv',
+                                dtype={
+                                    'track_id': 'int32',
+                                    'artist_id': 'int32'
+                                }).set_index('track_id')
+
+df_users = pd.read_csv('evalrs_dataset_KDD23/evalrs_users.csv',
+                            dtype={
+                                'user_id': 'int32',
+                                'playcount': 'int32',
+                                'country_id': 'int32',
+                                'timestamp': 'int32',
+                                'age': 'int32',
+                            })
+
+"""
+    Here we would normally train a model, but we just return random predictions.
+"""
+my_df_model = EvalRSSimpleModel(df_tracks, top_k=10)
+df_predictions = my_df_model.predict(df_users)
+# build a mock dataset for the golden standard
+all_tracks = df_tracks.index.values
+
+df_dataset = pd.DataFrame(
+    {
+        'track_id': [choice(all_tracks) for _ in range(len(df_predictions))]
+    }
+)
+
+"""
+    Here we use RecList to run the evaluation.
+"""
+
+# initialize with everything
+cdf = DFSessionRecList(
+    dataset=df_events,
+    model_name="myDataFrameRandomModel",
+    predictions=df_predictions,
+    # I can specify the gold standard here, or doing it in the init of course
+    y_test=df_dataset,
+    logger=LOGGER.NEPTUNE,
+    metadata_store=METADATA_STORE.LOCAL,
+    bucket=os.environ["S3_BUCKET"],
+    NEPTUNE_KEY=os.environ["NEPTUNE_KEY"],
+    NEPTUNE_PROJECT_NAME=os.environ["NEPTUNE_PROJECT_NAME"],
+)
+
+# run reclist
+cdf(verbose=True)
diff --git a/examples/movielens_similar_rec.py b/examples/movielens_similar_rec.py
deleted file mode 100644
index fc3c6ed..0000000
--- a/examples/movielens_similar_rec.py
+++ /dev/null
@@ -1,38 +0,0 @@
-"""
-    This is a sample script showing how to get started with reclist. In particular,
-    we are targeting a "similar items" use cases, such as for example a item recommender:
-    if a shopper is browsing a product listing and clicked on item X, what other item
-    might they be interested in that is similar in type to item X?
-
-    We train a simple, yet effective prod2vec baseline model (https://arxiv.org/abs/2007.14906),
-    re-using for convenience a "training embedding" function already implemented by recsys. The script
-    show how easy it is to run behavioral tests on a target dataset, in this case a wrapper around a large
-    dataset widely used for benchmarking recommendation systems (MovieLens 25M dataset: https://grouplens.org/datasets/movielens/)
-
-    If you want to run your own model on the same dataset and the same behavioral test suite, just implement
-    your model prediction using the RecModel, and pass it to the RecList.
-"""
-from reclist.datasets import MovieLensDataset
-from reclist.recommenders.prod2vec import MovieLensP2VRecModel
-from reclist.reclist import MovieLensSimilarItemRecList
-
-
-if __name__ == "__main__":
-
-    # get the MovieLens 25M dataset as a RecDataset object
-    movielens_dataset = MovieLensDataset()
-
-    # re-use a skip-gram model from reclist to train a latent product space, to be used
-    # (through knn) to build a recommender
-    model = MovieLensP2VRecModel()
-    model.train(movielens_dataset.x_train)
-
-    # instantiate rec_list object, prepared with standard quantitative tests
-    # and sensible behavioral tests (check the paper for details!)
-    rec_list = MovieLensSimilarItemRecList(
-        model=model,
-        dataset=movielens_dataset
-    )
-
-    # invoke rec_list to run tests
-    rec_list(verbose=True)
diff --git a/examples/spotify_session_rec.py b/examples/spotify_session_rec.py
deleted file mode 100644
index b7617bd..0000000
--- a/examples/spotify_session_rec.py
+++ /dev/null
@@ -1,35 +0,0 @@
-"""
-    This is a sample script showing how to get started with reclist. In particular,
-    we are targeting the use case of session-based recommendations.
-
-    We train a simple, yet effective prod2vec baseline model (https://arxiv.org/abs/2007.14906),
-    re-using for convenience a "training embedding" function already implemented by recsys. The script
-    shows how easy it is to run behavioral tests on a target dataset, in this case a wrapper around a large
-    music dataset (the Spotify Million Playlist dataset:
-    https://www.aicrowd.com/challenges/spotify-million-playlist-dataset-challenge).
-
-    If you want to run your own model on the same dataset and the same behavioral test suite, just implement
-    your model prediction using the RecModel, and pass it to the RecList.
-"""
-from reclist.datasets import SpotifyDataset
-from reclist.recommenders.prod2vec import SpotifyP2VRecModel
-from reclist.reclist import SpotifySessionRecList
-
-if __name__ == "__main__":
-
-    # get the Spotify million playlist dataset as a RecDataset object
-    spotify_dataset = SpotifyDataset()
-
-    # re-use a skip-gram model from reclist to train a latent product space, to be used
-    # (through knn) to build a recommender
-    model = SpotifyP2VRecModel()
-    model.train(spotify_dataset.x_train)
-
-    # instantiate rec_list object, prepared with standard quantitative tests
-    # and sensible behavioral tests (check the paper for details!)
-    rec_list = SpotifySessionRecList(
-        model=model,
-        dataset=spotify_dataset
-    )
-    # invoke rec_list to run tests
-    rec_list(verbose=True)
diff --git a/images/reclist.png b/images/reclist.png
new file mode 100644
index 0000000..7aa0efb
Binary files /dev/null and b/images/reclist.png differ
diff --git a/reclist/__init__.py b/reclist/__init__.py
index 02909f1..77b562c 100644
--- a/reclist/__init__.py
+++ b/reclist/__init__.py
@@ -1,5 +1,6 @@
 """Top-level package for RecList."""
 
+
 __author__ = """RecList"""
 __email__ = ''
 __version__ = '0.3.1'
diff --git a/reclist/abstractions.py b/reclist/abstractions.py
deleted file mode 100644
index 4514f40..0000000
--- a/reclist/abstractions.py
+++ /dev/null
@@ -1,245 +0,0 @@
-
-from datetime import datetime
-import os
-from abc import ABC, abstractmethod
-from functools import wraps
-from pathlib import Path
-import time
-import json
-from reclist.utils.train_w2v import train_embeddings
-from reclist.current import current
-
-class RecDataset(ABC):
-    """
-    Implements an abstract class for the dataset
-    """
-    def __init__(self, force_download=False):
-        """
-        :param force_download: allows to force the download of the dataset in case it is needed.
-        :type: force_download: bool, optional
-        """
-        self._x_train = None
-        self._y_train = None
-        self._x_test = None
-        self._y_test = None
-        self._catalog = None
-        self.force_download = force_download
-        self.load()
-
-    @abstractmethod
-    def load(self):
-        """
-        Abstract method that should implement dataset loading
-        :return:
-        """
-        return
-
-    @property
-    def x_train(self):
-        return self._x_train
-
-    @property
-    def y_train(self):
-        return self._y_train
-
-    @property
-    def x_test(self):
-        return self._x_test
-
-    @property
-    def y_test(self):
-        return self._y_test
-
-    @property
-    def catalog(self):
-        return self._catalog
-
-
-class RecModel(ABC):
-    """
-    Abstract class for recommendation model
-    """
-
-    def __init__(self, model=None):
-        """
-
-        :param model: a model that can be used in the predict function
-        """
-        self._model = model
-
-    @abstractmethod
-    def predict(self, prediction_input: list, *args, **kwargs):
-        """
-        The predict function should implement the behaviour of the model at inference time.
-
-        :param prediction_input: the input that is used to to do the prediction
-        :param args:
-        :param kwargs:
-        :return:
-        """
-        return NotImplementedError
-
-    @property
-    def model(self):
-        return self._model
-
-
-def rec_test(test_type: str):
-    """
-    Rec test decorator
-    """
-    def decorator(f):
-        @wraps(f)
-        def w(*args, **kwargs):
-            return f(*args, **kwargs)
-
-        # add attributes to f
-        w.is_test = True
-        w.test_type = test_type
-        try:
-            w.test_desc = f.__doc__.lstrip().rstrip()
-        except:
-            w.test_desc = ""
-        try:
-            # python 3
-            w.name = w.__name__
-        except:
-            # python 2
-            w.name = w.__func__.func_name
-        return w
-
-    return decorator
-
-
-class RecList(ABC):
-    META_DATA_FOLDER = '.reclist'
-
-    def __init__(self, model: RecModel, dataset: RecDataset, y_preds: list = None):
-        """
-        :param model:
-        :param dataset:
-        :param y_preds:
-        """
-
-        self.name = self.__class__.__name__
-        self._rec_tests = self.get_tests()
-        self._x_train = dataset.x_train
-        self._y_train = dataset.y_train
-        self._x_test = dataset.x_test
-        self._y_test = dataset.y_test
-        self._y_preds = y_preds if y_preds else model.predict(dataset.x_test)
-        self.rec_model = model
-        self.product_data = dataset.catalog
-        self._test_results = []
-        self._test_data = {}
-        self._dense_repr = {}
-
-        assert len(self._y_test) == len(self._y_preds)
-
-    def train_dense_repr(self, type_name: str, type_fn):
-        """
-        Train a dense representation over a type of meta-data & store into object
-        """
-        # type_fn: given a SKU returns some type i.e. brand
-        x_train_transformed = [[type_fn(e) for e in session if type_fn(e)] for session in self._x_train]
-        wv = train_embeddings(x_train_transformed)
-        # store a dict
-        self._dense_repr[type_name] = {word: list(wv.get_vector(word)) for word in wv.key_to_index}
-
-    def get_tests(self):
-        """
-        Helper to extract methods decorated with rec_test
-        """
-
-        nodes = {}
-        for _ in self.__dir__():
-            if not hasattr(self,_):
-                continue
-            func = getattr(self, _)
-            if hasattr(func, 'is_test'):
-                nodes[func.name] = func
-
-        return nodes
-
-    def __call__(self, verbose=True, *args, **kwargs):
-        run_epoch_time_ms = round(time.time() * 1000)
-        # create datastore
-        current._report_path = os.path.join(self.META_DATA_FOLDER,
-                                        self.name,
-                                        self.rec_model.__class__.__name__,
-                                        str(run_epoch_time_ms))
-
-        Path(os.path.join(current.report_path, 'artifacts')).mkdir(parents=True, exist_ok=True)
-        Path(os.path.join(current.report_path, 'results')).mkdir(parents=True, exist_ok=True)
-        Path(os.path.join(current.report_path, 'plots')).mkdir(parents=True, exist_ok=True)
-
-        # iterate through tests
-        for test_func_name, test in self._rec_tests.items():
-            test_result = test(*args, **kwargs)
-            # we could store the results in the test function itself
-            # test.__func__.test_result = test_result
-            self._test_results.append({
-                'test_name': test.test_type,
-                'description': test.test_desc,
-                'test_result': test_result}
-            )
-            if verbose:
-                print("============= TEST RESULTS ===============")
-                print("Test Type        : {}".format(test.test_type))
-                print("Test Description : {}".format(test.test_desc))
-                print("Test Result      : {}\n".format(test_result))
-        # at the end, we dump it locally
-        if verbose:
-            print("Generating reports at {}".format(datetime.utcnow()))
-        self.generate_report(run_epoch_time_ms)
-
-    def generate_report(self, epoch_time_ms: int):
-        # create path first: META_DATA_FOLDER / RecList / Model / Run Time
-        report_path = os.path.join(
-            self.META_DATA_FOLDER,
-            self.name,
-            self.rec_model.__class__.__name__,
-            str(epoch_time_ms)
-        )
-        # now, dump results
-        self.dump_results_to_json(self._test_results, report_path, epoch_time_ms)
-        # now, store artifacts
-        self.store_artifacts(report_path)
-
-    def store_artifacts(self, report_path: str):
-        target_path = os.path.join(current.report_path, 'artifacts')
-        # store predictions
-        with open(os.path.join(target_path, 'model_predictions.json'), 'w') as f:
-            json.dump({
-                'x_test': self._x_test,
-                'y_test': self._y_test,
-                'y_preds': self._y_preds
-            }, f)
-
-    def dump_results_to_json(self, test_results: list, report_path: str, epoch_time_ms: int):
-        target_path = os.path.join(report_path, 'results')
-        # make sure the folder is there, with all intermediate parents
-        Path(target_path).mkdir(parents=True, exist_ok=True)
-        report = {
-            'metadata': {
-                'run_time': epoch_time_ms,
-                'model_name': self.rec_model.__class__.__name__,
-                'reclist': self.name,
-                'tests': list(self._rec_tests.keys())
-            },
-            'data': test_results
-        }
-        with open(os.path.join(target_path, 'report.json'), 'w') as f:
-            json.dump(report, f)
-
-    @property
-    def test_results(self):
-        return self._test_results
-
-    @property
-    def test_data(self):
-        return self._test_data
-
-    @property
-    def rec_tests(self):
-        return self._rec_tests
diff --git a/reclist/charts.py b/reclist/charts.py
new file mode 100644
index 0000000..bb96c42
--- /dev/null
+++ b/reclist/charts.py
@@ -0,0 +1,8 @@
+
+from enum import Enum
+
+
+class CHART_TYPE(Enum):
+    SCALAR = 'scalar'
+    BARS = 'bars'
+    BINS = 'bins'
diff --git a/reclist/current.py b/reclist/current.py
deleted file mode 100644
index b3dbc3c..0000000
--- a/reclist/current.py
+++ /dev/null
@@ -1,10 +0,0 @@
-class Current():
-    def __init__(self):
-        self._report_path = None
-
-    @property
-    def report_path(self):
-        return self._report_path
-
-
-current = Current()
diff --git a/reclist/datasets.py b/reclist/datasets.py
deleted file mode 100644
index ccb3e34..0000000
--- a/reclist/datasets.py
+++ /dev/null
@@ -1,106 +0,0 @@
-import json
-import tempfile
-import zipfile
-import os
-from reclist.abstractions import RecDataset
-from reclist.utils.config import *
-
-
-class MovieLensDataset(RecDataset):
-    """
-    MovieLens 25M Dataset
-
-    Reference: https://files.grouplens.org/datasets/movielens/ml-25m-README.html
-    """
-
-    def __init__(self, **kwargs):
-        super().__init__(**kwargs)
-
-    def load(self):
-        cache_dir = get_cache_directory()
-        filepath = os.path.join(cache_dir, "movielens_25m.zip")
-
-        if not os.path.exists(filepath) or self.force_download:
-            download_with_progress(MOVIELENS_DATASET_S3_URL, filepath)
-
-        with tempfile.TemporaryDirectory() as temp_dir:
-            with zipfile.ZipFile(filepath, "r") as zip_file:
-                zip_file.extractall(temp_dir)
-            with open(os.path.join(temp_dir, "dataset.json")) as f:
-                data = json.load(f)
-
-        self._x_train = data["x_train"]
-        self._y_train = None
-        self._x_test = data["x_test"]
-        self._y_test = data["y_test"]
-        self._catalog = self._convert_catalog_keys(data["catalog"])
-        
-    def _convert_catalog_keys(self, catalog):
-        """
-        Convert catalog keys from string to integer type
-
-        JSON encodes all keys to strings, so the catalog dictionary
-        will be loaded up string representation of movie IDs.
-        """
-        converted_catalog = {}
-        for k, v in catalog.items():
-            converted_catalog[int(k)] = v
-        return converted_catalog
-
-class CoveoDataset(RecDataset):
-    """
-    Coveo SIGIR data challenge dataset
-    """
-    def __init__(self, **kwargs):
-        super().__init__(**kwargs)
-
-    def load(self):
-        cache_directory = get_cache_directory()
-        filename = os.path.join(cache_directory, "coveo_sigir.zip")  # TODO: make var somewhere
-
-        if not os.path.exists(filename) or self.force_download:
-            download_with_progress(COVEO_INTERACTION_DATASET_S3_URL, filename)
-
-        with tempfile.TemporaryDirectory() as temp_dir:
-            with zipfile.ZipFile(filename, 'r') as zip_ref:
-                zip_ref.extractall(temp_dir)
-            with open(os.path.join(temp_dir, 'dataset.json')) as f:
-                data = json.load(f)
-
-        self._x_train = data["x_train"]
-        self._y_train = None
-        self._x_test = data["x_test"]
-        self._y_test = data["y_test"]
-        self._catalog = data["catalog"]
-
-class SpotifyDataset(RecDataset):
-
-    def __init__(self, **kwargs):
-        super().__init__(**kwargs)
-
-    def load(self):
-        data = self.load_spotify_playlist_dataset()
-        self._x_train = data["train"]
-        self._y_train = None
-        self._x_test = data['test']
-        self._y_test = None
-        self._catalog = data["catalog"]
-
-        # generate NEP dataset here for now
-        test_pairs = [(playlist[:-1], [playlist[-1]])  for playlist in self._x_test if len(playlist) > 1]
-        self._x_test, self._y_test = zip(*test_pairs)
-
-    def load_spotify_playlist_dataset(self):
-
-        cache_directory = get_cache_directory()
-        filename = os.path.join(cache_directory, "small_spotify_playlist.zip")   # TODO: make var somewhere
-
-        if not os.path.exists(filename) or self.force_download:
-            download_with_progress(SPOTIFY_PLAYLIST_DATASET_S3_URL, filename)
-
-        with tempfile.TemporaryDirectory() as temp_dir:
-            with zipfile.ZipFile(filename, 'r') as zip_ref:
-                zip_ref.extractall(temp_dir)
-            with open(os.path.join(temp_dir, 'dataset.json')) as f:
-                data = json.load(f)
-        return data
diff --git a/reclist/logs.py b/reclist/logs.py
new file mode 100644
index 0000000..63a23a1
--- /dev/null
+++ b/reclist/logs.py
@@ -0,0 +1,113 @@
+import os
+from enum import Enum
+from abc import ABC, abstractmethod
+
+
+class LOGGER(Enum):
+    LOCAL = 1
+    COMET = 2
+    NEPTUNE = 3
+
+
+class RecLogger(ABC):
+
+    def __init__(self, *args, **kwargs):
+        pass
+
+    @abstractmethod
+    def write(self, label, value):
+        pass
+
+    @abstractmethod
+    def save_plot(self, name, fig, *args, **kwargs):
+        pass
+
+
+class LocalLogger(RecLogger):
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        return
+
+    def write(self, label, value):
+        from rich import print
+        print(f"[italic red]{label}[/italic red]:{value}")
+
+        return
+
+    def save_plot(self, name, fig, *args, **kwargs):
+        pass
+
+
+class NeptuneLogger(RecLogger):
+
+    def __init__(self, *args, **kwargs):
+        """
+
+        In order of priority, use first the kwargs, then the env variables
+
+        """
+        super().__init__(*args, **kwargs)
+        import neptune
+
+        api_key = kwargs.get("NEPTUNE_KEY", os.environ["NEPTUNE_KEY"])
+        project_name = kwargs.get("NEPTUNE_PROJECT_NAME", os.environ["NEPTUNE_PROJECT_NAME"])
+
+        self.experiment = neptune.init_run(
+            project=project_name,
+            api_token=api_key,
+        )
+
+    def write(self, label, value):
+        if isinstance(value, float):
+            self.experiment[label] = value
+
+    def save_plot(self, name, fig, *args, **kwargs):
+        import tempfile
+        with tempfile.NamedTemporaryFile() as temp:
+            file_name = temp.name + ".png"
+            fig.savefig(file_name)
+            self.experiment[name].upload(file_name)
+
+
+class CometLogger(RecLogger):
+
+    def __init__(self, *args, **kwargs):
+        """
+
+        In order of priority, use first the kwargs, then the env variables
+
+        """
+        super().__init__(*args, **kwargs)
+        from comet_ml import Experiment
+
+        api_key = kwargs.get("COMET_KEY", os.environ["COMET_KEY"])
+        project_name = kwargs.get("COMET_PROJECT_NAME", os.environ["COMET_PROJECT_NAME"])
+        workspace = kwargs.get("COMET_WORKSPACE", os.environ["COMET_WORKSPACE"])
+
+        # set up the experiment
+        self.experiment = Experiment(
+            api_key=api_key,
+            project_name=project_name,
+            workspace=workspace,
+        )
+
+    def write(self, label, value):
+        if isinstance(value, float):
+            self.experiment.log_metric(label, value)
+
+    def save_plot(self, name, fig, *args, **kwargs):
+        import tempfile
+        with tempfile.NamedTemporaryFile() as temp:
+            file_name = temp.name + ".png"
+            fig.savefig(file_name)
+            self.experiment.log_image(file_name, name=name)
+
+
+def logger_factory(label) -> RecLogger:
+    if label == LOGGER.COMET:
+        return CometLogger
+    elif label == LOGGER.NEPTUNE:
+        return NeptuneLogger
+    else:
+        return LocalLogger
diff --git a/reclist/metadata.py b/reclist/metadata.py
new file mode 100644
index 0000000..0d5ebec
--- /dev/null
+++ b/reclist/metadata.py
@@ -0,0 +1,73 @@
+import json
+import os
+import time
+from enum import Enum
+from abc import ABC, abstractmethod
+from pathlib import Path
+from functools import wraps
+import pandas as pd
+import numpy as np
+
+class METADATA_STORE(Enum):
+    LOCAL = 1
+    S3 = 2
+
+
+class MetaStore(ABC):
+
+    def __init__(self, *args, **kwargs):
+        pass
+
+    @abstractmethod
+    def write_file(self, path, data, is_json=False):
+        pass
+
+
+def metadata_store_factory(label) -> MetaStore:
+    if label == METADATA_STORE.S3:
+        return S3MetaStore
+    else:
+        return LocalMetaStore
+
+
+class LocalMetaStore(MetaStore):
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        return
+
+    def write_file(self, path, data, is_json=False):
+        if is_json:
+            with open(path, "w") as f:
+                json.dump(data, f, indent=2)
+        else:
+            with open(path, "w") as f:
+                f.write(data)
+
+        return
+
+
+class S3MetaStore(MetaStore):
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        return
+
+    def write_file(self, path, data, is_json=False):
+        """
+        We use s3fs to write to s3 - note: credentials are the default one
+        locally stored in ~/.aws/credentials
+
+        https://s3fs.readthedocs.io/en/latest/
+        """
+        import s3fs
+
+        s3 = s3fs.S3FileSystem(anon=False)
+        if is_json:
+            with s3.open(path, 'wb') as f:
+                f.write(json.dumps(data).encode('utf-8'))
+        else:
+            with s3.open(path, 'wb') as f:
+                f.write(data)
+
+        return
diff --git a/reclist/metrics/distance_metrics.py b/reclist/metrics/distance_metrics.py
index c89c1d4..9bc9539 100644
--- a/reclist/metrics/distance_metrics.py
+++ b/reclist/metrics/distance_metrics.py
@@ -1,25 +1,38 @@
-from scipy.spatial.distance import cosine
-import matplotlib.pyplot as plt
-from reclist.current import current
-import os
 import json
-import networkx as nx
-from networkx.algorithms.shortest_paths.generic import shortest_path
+import os
 from statistics import mean
-from reclist.metrics.standard_metrics import sample_misses_at_k, sample_hits_at_k
+
+import matplotlib.pyplot as plt
+import networkx as nx
 import numpy as np
+from networkx.algorithms.shortest_paths.generic import shortest_path
+from scipy.spatial.distance import cosine
+
+from reclist.current import current
+from reclist.metrics.standard_metrics import sample_hits_at_k, sample_misses_at_k
+
+
+def similarity_item(function, item1, item2):
+    return function(item1, item2)
 
 def error_by_cosine_distance(model, y_test, y_preds, k=3, bins=25, debug=False):
-    if not(hasattr(model.__class__, 'get_vector') and callable(getattr(model.__class__, 'get_vector'))):
-        error_msg = "Error : Model {} does not support retrieval of vector embeddings".format(model.__class__)
+    if not (
+        hasattr(model.__class__, "get_vector")
+        and callable(getattr(model.__class__, "get_vector"))
+    ):
+        error_msg = (
+            "Error : Model {} does not support retrieval of vector embeddings".format(
+                model.__class__
+            )
+        )
         print(error_msg)
         return error_msg
     misses = sample_misses_at_k(y_preds, y_test, k=k, size=-1)
     cos_distances = []
     for m in misses:
-        if m['Y_PRED']:
-            vector_test = model.get_vector(m['Y_TEST'][0])
-            vector_pred = model.get_vector(m['Y_PRED'][0])
+        if m["Y_PRED"]:
+            vector_test = model.get_vector(m["Y_TEST"][0])
+            vector_pred = model.get_vector(m["Y_PRED"][0])
             if vector_pred and vector_test:
                 cos_dist = cosine(vector_pred, vector_test)
                 cos_distances.append(cos_dist)
@@ -30,29 +43,37 @@ def error_by_cosine_distance(model, y_test, y_preds, k=3, bins=25, debug=False):
     # debug / viz
     if debug:
         plt.hist(cos_distances, bins=bins)
-        plt.title('dist over cosine distance prod space')
-        plt.savefig(os.path.join(current.report_path,
-                                 'plots',
-                                 'distance_to_predictions.png'))
+        plt.title("dist over cosine distance prod space")
+        plt.savefig(
+            os.path.join(current.report_path, "plots", "distance_to_predictions.png")
+        )
         plt.clf()
         # plt.show()
 
-    return {'mean': np.mean(cos_distances), 'histogram': histogram}
+    return {"mean": np.mean(cos_distances), "histogram": histogram}
+
 
 def distance_to_query(model, x_test, y_test, y_preds, k=3, bins=25, debug=False):
-    if not(hasattr(model.__class__, 'get_vector') and callable(getattr(model.__class__, 'get_vector'))):
-        error_msg = "Error : Model {} does not support retrieval of vector embeddings".format(model.__class__)
+    if not (
+        hasattr(model.__class__, "get_vector")
+        and callable(getattr(model.__class__, "get_vector"))
+    ):
+        error_msg = (
+            "Error : Model {} does not support retrieval of vector embeddings".format(
+                model.__class__
+            )
+        )
         print(error_msg)
         return error_msg
     misses = sample_misses_at_k(y_preds, y_test, x_test=x_test, k=k, size=-1)
     x_to_y_cos = []
     x_to_p_cos = []
     for m in misses:
-        if m['Y_PRED']:
-            vector_x = model.get_vector(m['X_TEST'][0])
-            vector_y = model.get_vector(m['Y_TEST'][0])
-            vectors_p = [model.get_vector(_) for _ in m['Y_PRED']]
-            c_dists =[]
+        if m["Y_PRED"]:
+            vector_x = model.get_vector(m["X_TEST"][0])
+            vector_y = model.get_vector(m["Y_TEST"][0])
+            vectors_p = [model.get_vector(_) for _ in m["Y_PRED"]]
+            c_dists = []
             if not vector_x or not vector_y:
                 continue
             x_to_y_cos.append(cosine(vector_x, vector_y))
@@ -75,38 +96,39 @@ def distance_to_query(model, x_test, y_test, y_preds, k=3, bins=25, debug=False)
     if debug:
         plt.hist(x_to_y_cos, bins=bins, alpha=0.5)
         plt.hist(x_to_p_cos, bins=bins, alpha=0.5)
-        plt.title('distribution of distance to input')
-        plt.legend(['Distance from Input to Label',
-                    'Distance from Input to Label'],
-                   loc='upper right')
+        plt.title("distribution of distance to input")
+        plt.legend(
+            ["Distance from Input to Label", "Distance from Input to Label"],
+            loc="upper right",
+        )
         # plt.show()
-        plt.savefig(os.path.join(current.report_path,
-                                 'plots',
-                                 'distance_to_query.png'))
+        plt.savefig(os.path.join(current.report_path, "plots", "distance_to_query.png"))
         plt.clf()
 
     return {
-        'histogram_x_to_y': h_xy,
-        'histogram_x_to_p': h_xp,
-        'raw_distances_x_to_y': x_to_y_cos,
-        'raw_distances_x_to_p': x_to_p_cos,
+        "histogram_x_to_y": h_xy,
+        "histogram_x_to_p": h_xp,
+        "raw_distances_x_to_y": x_to_y_cos,
+        "raw_distances_x_to_p": x_to_p_cos,
     }
 
 
-
 def shortest_path_length():
     pass
 
 
-get_nodes = lambda nodes, ancestors="": [] if not nodes else ['_'.join([ancestors, nodes[0]])] + \
-                                                             get_nodes(nodes[1:], '_'.join([ancestors, nodes[0]]))
+get_nodes = (
+    lambda nodes, ancestors="": []
+    if not nodes
+    else ["_".join([ancestors, nodes[0]])]
+    + get_nodes(nodes[1:], "_".join([ancestors, nodes[0]]))
+)
 
 
 def graph_distance_test(y_test, y_preds, product_data, k=3):
     path_lengths = []
     misses = sample_misses_at_k(y_preds, y_test, k=k, size=-1)
-    for _y, _y_p in zip([_['Y_TEST'] for _ in misses],
-                        [_['Y_PRED'] for _ in misses]):
+    for _y, _y_p in zip([_["Y_TEST"] for _ in misses], [_["Y_PRED"] for _ in misses]):
         if not _y_p:
             continue
         _y_sku = _y[0]
@@ -114,15 +136,20 @@ def graph_distance_test(y_test, y_preds, product_data, k=3):
 
         if _y_sku not in product_data or _y_p_sku not in product_data:
             continue
-        if not product_data[_y_sku]['CATEGORIES'] or not product_data[_y_p_sku]['CATEGORIES']:
+        if (
+            not product_data[_y_sku]["CATEGORIES"]
+            or not product_data[_y_p_sku]["CATEGORIES"]
+        ):
             continue
         # extract graph information
-        catA = json.loads(product_data[_y_sku]['CATEGORIES'])
-        catB = json.loads(product_data[_y_p_sku]['CATEGORIES'])
+        catA = json.loads(product_data[_y_sku]["CATEGORIES"])
+        catB = json.loads(product_data[_y_p_sku]["CATEGORIES"])
         catA_nodes = [get_nodes(c) for c in catA]
         catB_nodes = [get_nodes(c) for c in catB]
         all_nodes = list(set([n for c in catA_nodes + catB_nodes for n in c]))
-        all_edges = [(n1, n2) for c in catA_nodes + catB_nodes for n1, n2 in zip(c[:-1], c[1:])]
+        all_edges = [
+            (n1, n2) for c in catA_nodes + catB_nodes for n1, n2 in zip(c[:-1], c[1:])
+        ]
         all_edges = list(set(all_edges))
 
         # build graph
@@ -134,31 +161,33 @@ def graph_distance_test(y_test, y_preds, product_data, k=3):
         cat1_leaves = [c[-1] for c in catA_nodes]
         cat2_leaves = [c[-1] for c in catB_nodes]
 
-        all_paths = [shortest_path(G, c1_l, c2_l) for c1_l in cat1_leaves for c2_l in cat2_leaves]
+        all_paths = [
+            shortest_path(G, c1_l, c2_l) for c1_l in cat1_leaves for c2_l in cat2_leaves
+        ]
         s_path = min(all_paths, key=len)
         s_path_len = len(s_path) - 1
         path_lengths.append(s_path_len)
 
     histogram = np.histogram(path_lengths, bins=np.arange(0, max(path_lengths)))
     histogram = (histogram[0].tolist(), histogram[1].tolist())
-    return {'mean': mean(path_lengths), 'hist': histogram}
-
+    return {"mean": mean(path_lengths), "hist": histogram}
 
 
-def generic_cosine_distance(embeddings: dict,
-                            type_fn,
-                            y_test,
-                            y_preds,
-                            k=10,
-                            bins=25,
-                            debug=False):
+def generic_cosine_distance(
+    embeddings: dict, type_fn, y_test, y_preds, k=10, bins=25, debug=False
+):
 
     misses = sample_misses_at_k(y_preds, y_test, k=k, size=-1)
     cos_distances = []
     for m in misses:
-        if m['Y_TEST'] and m['Y_PRED'] and type_fn(m['Y_TEST'][0]) and type_fn(m['Y_PRED'][0]):
-            vector_test = embeddings.get(type_fn(m['Y_TEST'][0]), None)
-            vector_pred = embeddings.get(type_fn(m['Y_PRED'][0]), None)
+        if (
+            m["Y_TEST"]
+            and m["Y_PRED"]
+            and type_fn(m["Y_TEST"][0])
+            and type_fn(m["Y_PRED"][0])
+        ):
+            vector_test = embeddings.get(type_fn(m["Y_TEST"][0]), None)
+            vector_pred = embeddings.get(type_fn(m["Y_PRED"][0]), None)
             if vector_pred and vector_test:
                 cos_dist = cosine(vector_pred, vector_test)
                 cos_distances.append(cos_dist)
@@ -170,10 +199,10 @@ def generic_cosine_distance(embeddings: dict,
     # debug / viz
     if debug:
         plt.hist(cos_distances, bins=bins)
-        plt.title('cosine distance misses')
-        plt.savefig(os.path.join(current.report_path,
-                                 'plots',
-                                 'cosine_distance_over_type.png'))
+        plt.title("cosine distance misses")
+        plt.savefig(
+            os.path.join(current.report_path, "plots", "cosine_distance_over_type.png")
+        )
         plt.clf()
 
-    return {'mean': np.mean(cos_distances), 'histogram': histogram}
+    return {"mean": np.mean(cos_distances), "histogram": histogram}
diff --git a/reclist/metrics/hits.py b/reclist/metrics/hits.py
index 8b351e2..884e52c 100644
--- a/reclist/metrics/hits.py
+++ b/reclist/metrics/hits.py
@@ -1,11 +1,16 @@
-from collections import Counter, defaultdict
-import numpy as np
 import math
-from reclist.metrics.standard_metrics import hit_rate_at_k, sample_hits_at_k, sample_misses_at_k
-from collections import defaultdict
 import os
+from collections import Counter, defaultdict
+
 import matplotlib.pyplot as plt
+import numpy as np
+
 from reclist.current import current
+from reclist.metrics.standard_metrics import (
+    hit_rate_at_k,
+    sample_hits_at_k,
+    sample_misses_at_k,
+)
 
 
 def hits_distribution_by_rating(y_test, y_preds, debug=False):
@@ -35,18 +40,19 @@ def hits_distribution_by_rating(y_test, y_preds, debug=False):
         plt.bar(
             x_tick_idx,
             [hit_rate for hit_rate in hits_distribution_by_rating.values()],
-            align='center'
+            align="center",
         )
         plt.xticks(
             list(range(len(hits_distribution_by_rating))), x_tick_names, fontsize=10
         )
-        plt.savefig(os.path.join(current.report_path,
-                                'plots',
-                                'hit_distribution_rating.png'))
+        plt.savefig(
+            os.path.join(current.report_path, "plots", "hit_distribution_rating.png")
+        )
         plt.clf()
 
     return hits_distribution_by_rating
 
+
 def roundup(x: int):
     div = 10.0 ** (len(str(x)))
     return int(math.ceil(x / div)) * div
@@ -65,35 +71,36 @@ def hits_distribution(x_train, x_test, y_test, y_preds, k=3, debug=False):
     # round up to nearest place
     max_cnt = int(roundup(max_cnt))
     # generate log-bins
-    indices = np.logspace(1, np.log10(max_cnt), num=int(np.log10(max_cnt))).astype(np.int64)
+    indices = np.logspace(1, np.log10(max_cnt), num=int(np.log10(max_cnt))).astype(
+        np.int64
+    )
     indices = np.concatenate(([0], indices))
-    counts_per_bin = [[_ for i in range(low, high) for _ in hit_per_interaction_cnt[i]]
-                      for low, high in zip(indices[:-1], indices[1:])]
+    counts_per_bin = [
+        [_ for i in range(low, high) for _ in hit_per_interaction_cnt[i]]
+        for low, high in zip(indices[:-1], indices[1:])
+    ]
 
     histogram = [np.mean(counts) if counts else 0 for counts in counts_per_bin]
     count = [len(counts) for counts in counts_per_bin]
 
     if debug:
         # debug / visualization
-        plt.bar(indices[1:], histogram, width=-np.diff(indices) / 1.05, align='edge')
-        plt.xscale('log', base=10)
-        plt.title('HIT Distribution Across Product Frequency')
+        plt.bar(indices[1:], histogram, width=-np.diff(indices) / 1.05, align="edge")
+        plt.xscale("log", base=10)
+        plt.title("HIT Distribution Across Product Frequency")
         # plt.show()
-        plt.savefig(os.path.join(current.report_path, 'plots', 'hit_distribution.png'))
+        plt.savefig(os.path.join(current.report_path, "plots", "hit_distribution.png"))
         plt.clf()
 
     return {
-        'histogram': {int(k): v for k, v in zip(indices[1:], histogram)},
-        'counts': {int(k): v for k, v in zip(indices[1:], count)}
+        "histogram": {int(k): v for k, v in zip(indices[1:], histogram)},
+        "counts": {int(k): v for k, v in zip(indices[1:], count)},
     }
 
 
-def hits_distribution_by_slice(slices,
-                               y_test,
-                               y_preds,
-                               k=3,
-                               sample_size=3,
-                               debug=False):
+def hits_distribution_by_slice(
+    slices, y_test, y_preds, k=3, sample_size=3, debug=False
+):
     hit_rate_per_slice = defaultdict(dict)
     for slice_name, slice_idx in slices.items():
         # get predictions for slice
@@ -103,7 +110,7 @@ def hits_distribution_by_slice(slices,
         # TODO: We may want to allow for generic metric to be used here
         slice_hr = hit_rate_at_k(slice_y_preds, slice_y_test, k=k)
         # store results
-        hit_rate_per_slice[slice_name]['hit_rate'] = slice_hr
+        hit_rate_per_slice[slice_name]["hit_rate"] = slice_hr
         # hit_rate_per_slice[slice_name]['hits'] = sample_hits_at_k(slice_y_preds, slice_y_test, k=k, size=sample_size)
         # hit_rate_per_slice[slice_name]['misses'] = sample_misses_at_k(slice_y_preds, slice_y_test, k=k, size=sample_size)
 
@@ -112,11 +119,17 @@ def hits_distribution_by_slice(slices,
         x_tick_names = list(hit_rate_per_slice.keys())
         x_tick_idx = list(range(len(x_tick_names)))
         plt.figure(dpi=150)
-        plt.bar(x_tick_idx, [_['hit_rate'] for _ in hit_rate_per_slice.values()], align='center')
-        plt.xticks(list(range(len(hit_rate_per_slice))), x_tick_names, rotation=45, fontsize=5)
-        plt.savefig(os.path.join(current.report_path,
-                                 'plots',
-                                 'hit_distribution_slice.png'))
+        plt.bar(
+            x_tick_idx,
+            [_["hit_rate"] for _ in hit_rate_per_slice.values()],
+            align="center",
+        )
+        plt.xticks(
+            list(range(len(hit_rate_per_slice))), x_tick_names, rotation=45, fontsize=5
+        )
+        plt.savefig(
+            os.path.join(current.report_path, "plots", "hit_distribution_slice.png")
+        )
         plt.clf()
 
     # cast to normal dict
diff --git a/reclist/metrics/perturbation.py b/reclist/metrics/perturbation.py
index 2cd9b87..1494953 100644
--- a/reclist/metrics/perturbation.py
+++ b/reclist/metrics/perturbation.py
@@ -1,14 +1,13 @@
 import numpy as np
 
 
-def session_perturbation_test(model,
-                              x_test,
-                              y_preds,
-                              perturbation_fn,
-                              id_fn,
-                              k):
+def session_perturbation_test(model, x_test, y_preds, perturbation_fn, id_fn, k):
     # generate perturbations
-    perturbed_pairs = [(perturbation_fn(_x), _y_p) for _x, _y_p in zip(x_test, y_preds) if perturbation_fn(_x)]
+    perturbed_pairs = [
+        (perturbation_fn(_x), _y_p)
+        for _x, _y_p in zip(x_test, y_preds)
+        if perturbation_fn(_x)
+    ]
     # extract perturbed x and original y
     x_test_p, y_preds_o = zip(*perturbed_pairs)
     # make new predictions over perturbed inputs
diff --git a/reclist/metrics/price_homogeneity.py b/reclist/metrics/price_homogeneity.py
index d21659c..2ffd930 100644
--- a/reclist/metrics/price_homogeneity.py
+++ b/reclist/metrics/price_homogeneity.py
@@ -1,9 +1,14 @@
-import numpy as np
-import matplotlib.pyplot as plt
 import os
+
+import matplotlib.pyplot as plt
+import numpy as np
+
 from reclist.current import current
 
-def price_homogeneity_test(y_test, y_preds, product_data, price_sel_fn, bins=25, debug=True):
+
+def price_homogeneity_test(
+    y_test, y_preds, product_data, price_sel_fn, bins=25, debug=True
+):
     """
 
     :param y_test: test data
@@ -27,19 +32,14 @@ def price_homogeneity_test(y_test, y_preds, product_data, price_sel_fn, bins=25,
         if price_sel_fn(_y_info) and price_sel_fn(_y_pred_info):
             y_item_price = price_sel_fn(_y_info)
             pred_item_price = price_sel_fn(_y_pred_info)
-            price_diff = np.abs(np.log10(pred_item_price/y_item_price))
+            price_diff = np.abs(np.log10(pred_item_price / y_item_price))
             abs_log_price_diff.append(price_diff)
 
     histogram = np.histogram(abs_log_price_diff, bins=bins, density=False)
     histogram = (histogram[0].tolist(), histogram[1].tolist())
     if debug:
         plt.hist(abs_log_price_diff, bins=25)
-        plt.savefig(os.path.join(current.report_path,
-                                 'plots',
-                                 'price_homogeneity.png'))
+        plt.savefig(os.path.join(current.report_path, "plots", "price_homogeneity.png"))
         plt.clf()
 
-    return {
-        'mean': np.mean(abs_log_price_diff),
-        'histogram': histogram
-    }
+    return {"mean": np.mean(abs_log_price_diff), "histogram": histogram}
diff --git a/reclist/metrics/standard_metrics.py b/reclist/metrics/standard_metrics.py
index db1323a..967b870 100644
--- a/reclist/metrics/standard_metrics.py
+++ b/reclist/metrics/standard_metrics.py
@@ -1,7 +1,210 @@
-import random
+"""
+Implement standard recommender system retrieval metrics leveraging numpy/pandas backend.
+"""
+import collections
 import itertools
+import random
 import numpy as np
-import collections
+import pandas as pd
+from sklearn.metrics import accuracy_score
+from typing import Union
+
+
+def func_per_slice(y_test, y_pred, categories, func):
+    labels = set(categories)
+
+    c = []
+    global_score = func(y_test, y_pred)
+
+    for label in labels:
+        mask = np.array([True if k == label else False for k in categories])
+        local_score = func(y_test[mask], y_pred[mask])
+        c.append(np.abs(local_score - global_score))
+    return np.mean(c)
+
+def accuracy_per_slice(y_test, y_pred, categories):
+
+    y_test = np.array(y_test)
+    y_pred = np.array(y_pred)
+    return func_per_slice(y_test, y_pred, categories, accuracy_score)
+
+def hits_at_k(
+    y_pred: pd.DataFrame, 
+    y_test: pd.DataFrame,
+    k: int
+) -> Union[np.array, pd.DataFrame]:
+    """
+    Computes whether for the n-th test case, the m-th ground truth is a hit with the k-th prediction.
+
+    Parameters
+    ----------
+    y_pred: pd.DataFrame
+        Array of predictions with shape N x k (?)
+    y_test: pd.DataFrame
+        Array of ground truth with shape N x m
+    k: int
+        Index cut-off for prediciton 
+    
+    Returns
+    -------
+    out : np.array, pd.DataFrame
+         array indicicating whether for the n-th test case, the m-th ground truth is a hit with the k-th prediction
+
+    Examples
+    --------
+    >>> y_pred = pd.DataFrame([[1, 2, 4], [3, 6, 2]])
+    >>> y_test = pd.DataFrame([[5, 1, 0], [6, 2, 3]])
+    >>> hits_at_k(y_pred, y_test, k=2)
+    array([
+        [[False, False],
+         [True, False],
+         [False, False]],
+        [[False, True],
+         [False, False],
+         [True, False]]
+    ])
+    """
+    
+    y_test_mask = ~y_test.isna().values                # N x M
+
+    y_pred_mask = ~y_pred.isna().values[:, :k]  # N x k
+
+    y_test = y_test.values[:, :, None]        # N x M x 1
+    y_pred = y_pred.values[:, None, :k]       # N x 1 x k
+
+    hits = y_test == y_pred                   # N x M x k
+    hits = hits * y_test_mask[:, :, None]     # N x M x k
+    hits = hits * y_pred_mask[:, None, :]     # N x M x k
+
+    return hits
+
+
+def ranks_at_k(
+    y_pred: pd.DataFrame, 
+    y_test: pd.DataFrame,
+    k: int
+) -> Union[np.array, pd.DataFrame]:
+    """
+    Computes for every test case, n, the rank of the m-th ground truth label out of k predictions.
+    Rank is 0 if ground truth is not in prediction.
+
+    Parameters
+    ----------
+    y_pred: pd.DataFrame
+        Array of predictions with shape N x k (?)
+    y_test: pd.DataFrame
+        Array of ground truth with shape N x m
+    k: int
+        Index cut-off for prediciton 
+    
+    Returns
+    -------
+    out : np.array, pd.DataFrame
+         array indicicating for each test case n, the rank of the m-th ground truth label in k predictions.
+
+    Examples
+    --------
+    >>> y_pred = pd.DataFrame([[1, 2, 4], [3, 6, 2]])
+    >>> y_test = pd.DataFrame([[5, 1, 0], [6, 2, 3]])
+    >>> hits_at_k(y_pred, y_test, k=2)
+    array([[0, 1, 0],
+           [2, 0, 1]])
+
+    """
+    
+    hits = hits_at_k(y_pred, y_test, k)                     # N x M x k
+    ranks = hits * np.arange(1, k + 1, 1)[None, None, :]    # N x M x k
+    ranks = ranks.max(axis=2)                               # N x M
+    return ranks
+
+
+def misses_at_k(
+    y_pred: pd.DataFrame,
+    y_test: pd.DataFrame,
+    k: int
+) -> Union[np.array, pd.DataFrame]:
+    """
+    Computes whether for the n-th test case, the m-th ground truth is a miss with the k-th prediction.
+
+    Parameters
+    ----------
+    y_pred: pd.DataFrame
+        Array of predictions with shape N x k (?)
+    y_test: pd.DataFrame
+        Array of ground truth with shape N x m
+    k: int
+        Index cut-off for prediciton 
+    
+    Returns
+    -------
+    out : np.array, pd.DataFrame
+         array indicicating whether for the n-th test case, the m-th ground truth is a miss with the k-th prediction
+
+    Examples
+    --------
+    >>> y_pred = pd.DataFrame([[1, 2, 4], [3, 6, 2]])
+    >>> y_test = pd.DataFrame([[5, 1, 0], [6, 2, 3]])
+    >>> misses_at_k(y_pred, y_test, k=2)
+    array([
+        [[True, True],
+         [False, True],
+         [True, True]],
+        [[True, False],
+         [True, True],
+         [False, True]]
+    ])
+    """  
+
+    hits = hits_at_k(y_pred, y_test, k)  # N x M x k
+    return ~hits
+
+
+def hit_rate_at_k(
+    y_pred: pd.DataFrame,
+    y_test: pd.DataFrame,
+    k: int
+) -> float:
+    """
+    Computes the hit rate @ k
+
+    Parameters
+    ----------
+    y_pred: pd.DataFrame
+        Array of predictions with shape N x k (?)
+    y_test: pd.DataFrame
+        Array of ground truth with shape N x m
+    k: int
+        Index cut-off for prediciton 
+    
+    Returns
+    -------
+    out : float
+         hit rate @ k
+
+    Examples
+    --------
+    >>> y_pred = pd.DataFrame([[1, 2, 4], [3, 6, 2]])
+    >>> y_test = pd.DataFrame([[5, 1, 0], [1, 2, 0]])
+    >>> hit_rate_at_k(y_pred, y_test, k=2)
+    0.5
+    """
+
+    hits = hits_at_k(y_pred, y_test, k)  # N x M x k
+    hits = hits.max(axis=1)              # N x k
+    return hits.max(axis=1).mean()       # 1
+
+
+def rr_at_k(y_pred: pd.DataFrame, y_test: pd.DataFrame, k: int):
+    ranks = ranks_at_k(y_pred, y_test, k).astype(np.float64)  # N x M
+    reciprocal_ranks = np.reciprocal(ranks, out=ranks, where=ranks > 0)  # N x M
+    return reciprocal_ranks.max(axis=1)  # N
+
+
+def mrr_at_k(y_pred: pd.DataFrame, y_test: pd.DataFrame, k: int) -> float:
+    return rr_at_k(y_pred, y_test, k=k).mean()
+
+
+####################### LIST-WISE CODE #####################################
 
 
 def statistics(x_train, y_train, x_test, y_test, y_pred):
@@ -10,9 +213,9 @@ def statistics(x_train, y_train, x_test, y_test, y_pred):
     # num non-zero preds
     num_preds = len([p for p in y_pred if p])
     return {
-        'training_set__size': train_size,
-        'test_set_size': test_size,
-        'num_non_null_predictions': num_preds
+        "training_set__size": train_size,
+        "test_set_size": test_size,
+        "num_non_null_predictions": num_preds,
     }
 
 
@@ -21,11 +224,11 @@ def sample_hits_at_k(y_preds, y_test, x_test=None, k=3, size=3):
     for idx, (_p, _y) in enumerate(zip(y_preds, y_test)):
         if _y[0] in _p[:k]:
             hit_info = {
-                'Y_TEST': [_y[0]],
-                'Y_PRED': _p[:k],
+                "Y_TEST": [_y[0]],
+                "Y_PRED": _p[:k],
             }
             if x_test:
-                hit_info['X_TEST'] = [x_test[idx][0]]
+                hit_info["X_TEST"] = [x_test[idx][0]]
             hits.append(hit_info)
 
     if len(hits) < size or size == -1:
@@ -37,12 +240,12 @@ def sample_misses_at_k(y_preds, y_test, x_test=None, k=3, size=3):
     misses = []
     for idx, (_p, _y) in enumerate(zip(y_preds, y_test)):
         if _y[0] not in _p[:k]:
-            miss_info =  {
-                'Y_TEST': [_y[0]],
-                'Y_PRED': _p[:k],
+            miss_info = {
+                "Y_TEST": [_y[0]],
+                "Y_PRED": _p[:k],
             }
             if x_test:
-                miss_info['X_TEST'] = [x_test[idx][0]]
+                miss_info["X_TEST"] = [x_test[idx][0]]
             misses.append(miss_info)
 
     if len(misses) < size or size == -1:
@@ -52,10 +255,10 @@ def sample_misses_at_k(y_preds, y_test, x_test=None, k=3, size=3):
 
 def hit_rate_at_k_nep(y_preds, y_test, k=3):
     y_test = [[k] for k in y_test]
-    return hit_rate_at_k(y_preds, y_test, k=k)
+    return hit_rate_at_k_list(y_preds, y_test, k=k)
 
 
-def hit_rate_at_k(y_preds, y_test, k=3):
+def hit_rate_at_k_list(y_preds, y_test, k=3) -> float:
     hits = 0
     for _p, _y in zip(y_preds, y_test):
         if len(set(_p[:k]).intersection(set(_y))) > 0:
@@ -63,7 +266,7 @@ def hit_rate_at_k(y_preds, y_test, k=3):
     return hits / len(y_test)
 
 
-def mrr_at_k_nep(y_preds, y_test, k=3):
+def mrr_at_k_nep_list(y_preds, y_test, k=3):
     """
     Computes MRR
 
@@ -72,10 +275,10 @@ def mrr_at_k_nep(y_preds, y_test, k=3):
     :param k: top-k
     """
     y_test = [[k] for k in y_test]
-    return mrr_at_k(y_preds, y_test, k=k)
+    return mrr_at_k_list(y_preds, y_test, k=k)
 
 
-def mrr_at_k(y_preds, y_test, k=3):
+def mrr_at_k_list(y_preds, y_test, k=3):
     """
     Computes MRR
 
@@ -105,26 +308,34 @@ def coverage_at_k(y_preds, product_data, k=3):
 
 def popularity_bias_at_k(y_preds, x_train, k=3):
     # estimate popularity from training data
-    pop_map = collections.defaultdict(lambda : 0)
+    pop_map = collections.defaultdict(lambda: 0)
     num_interactions = 0
     for session in x_train:
         for event in session:
             pop_map[event] += 1
             num_interactions += 1
     # normalize popularity
-    pop_map = {k:v/num_interactions for k,v in pop_map.items()}
+    pop_map = {k: v / num_interactions for k, v in pop_map.items()}
     all_popularity = []
     for p in y_preds:
-        average_pop = sum(pop_map.get(_, 0.0) for _ in p[:k]) / len(p) if len(p) > 0 else 0
+        average_pop = (
+            sum(pop_map.get(_, 0.0) for _ in p[:k]) / len(p) if len(p) > 0 else 0
+        )
         all_popularity.append(average_pop)
     return sum(all_popularity) / len(y_preds)
 
 
 def precision_at_k(y_preds, y_test, k=3):
-    precision_ls = [len(set(_y).intersection(set(_p[:k]))) / len(_p[:k]) if _p else 1 for _p, _y in zip(y_preds, y_test)]
+    precision_ls = [
+        len(set(_y).intersection(set(_p[:k]))) / len(_p[:k]) if _p else 1
+        for _p, _y in zip(y_preds, y_test)
+    ]
     return np.average(precision_ls)
 
 
 def recall_at_k(y_preds, y_test, k=3):
-    recall_ls = [len(set(_y).intersection(set(_p[:k]))) / len(_y) if _y else 1 for _p, _y in zip(y_preds, y_test)]
+    recall_ls = [
+        len(set(_y).intersection(set(_p[:k]))) / len(_y) if _y else 1
+        for _p, _y in zip(y_preds, y_test)
+    ]
     return np.average(recall_ls)
diff --git a/reclist/reclist.py b/reclist/reclist.py
index 28d9aa5..242d640 100644
--- a/reclist/reclist.py
+++ b/reclist/reclist.py
@@ -1,287 +1,265 @@
-import collections
-from reclist.abstractions import RecList, rec_test
-from typing import List
-import random
-
-
-class CoveoCartRecList(RecList):
-
-    @rec_test(test_type='stats')
-    def basic_stats(self):
-        """
-        Basic statistics on training, test and prediction data
-        """
-        from reclist.metrics.standard_metrics import statistics
-        return statistics(self._x_train,
-                          self._y_train,
-                          self._x_test,
-                          self._y_test,
-                          self._y_preds)
-
-    @rec_test(test_type='price_homogeneity')
-    def price_test(self):
-        """
-        Measures the absolute log ratio of ground truth and prediction price
-        """
-        from reclist.metrics.price_homogeneity import price_homogeneity_test
-        return price_homogeneity_test(y_test=self.sku_only(self._y_test),
-                                      y_preds=self.sku_only(self._y_preds),
-                                      product_data=self.product_data,
-                                      price_sel_fn=lambda x: float(x['price_bucket'])
-                                                             if x['price_bucket']
-                                                             else None
-                                      )
-
-    @rec_test(test_type='Coverage@10')
-    def coverage_at_k(self):
-        """
-        Coverage is the proportion of all possible products which the RS
-        recommends based on a set of sessions
-        """
-        from reclist.metrics.standard_metrics import coverage_at_k
-        return coverage_at_k(self.sku_only(self._y_preds),
-                             self.product_data,
-                             k=10)
-
-    @rec_test(test_type='HR@10')
-    def hit_rate_at_k(self):
-        """
-        Compute the rate in which the top-k predictions contain the item to be predicted
-        """
-        from reclist.metrics.standard_metrics import hit_rate_at_k
-        return hit_rate_at_k(self.sku_only(self._y_preds),
-                             self.sku_only(self._y_test),
-                             k=10)
-
-    @rec_test(test_type='hits_distribution')
-    def hits_distribution(self):
-        """
-        Compute the distribution of hit-rate across product frequency in training data
-        """
-        from reclist.metrics.hits import hits_distribution
-        return hits_distribution(self.sku_only(self._x_train),
-                                 self.sku_only(self._x_test),
-                                 self.sku_only(self._y_test),
-                                 self.sku_only(self._y_preds),
-                                 k=10,
-                                 debug=True)
-
-    @rec_test(test_type='distance_to_query')
-    def dist_to_query(self):
-        """
-        Compute the distribution of distance from query to label and query to prediction
-        """
-        from reclist.metrics.distance_metrics import distance_to_query
-        return distance_to_query(self.rec_model,
-                                 self.sku_only(self._x_test),
-                                 self.sku_only(self._y_test),
-                                 self.sku_only(self._y_preds), k=10, bins=25, debug=True)
-
-    def sku_only(self, l: List[List]):
-        return [[e['product_sku'] for e in s] for s in l]
-
-
-class SpotifySessionRecList(RecList):
-
-    @rec_test(test_type='basic_stats')
-    def basic_stats(self):
-        """
-        Basic statistics on training, test and prediction data for Next Event Prediction
-        """
-        from reclist.metrics.standard_metrics import statistics
-        return statistics(self._x_train,
-                          self._y_train,
-                          self._x_test,
-                          self._y_test,
-                          self._y_preds)
-
-    @rec_test(test_type='HR@10')
-    def hit_rate_at_k(self):
-        """
-        Compute the rate at which the top-k predictions contain the item to be predicted
-        """
-        from reclist.metrics.standard_metrics import hit_rate_at_k
-        return hit_rate_at_k(self.uri_only(self._y_preds),
-                             self.uri_only(self._y_test),
-                             k=10)
-
-    @rec_test(test_type='perturbation_test')
-    def perturbation_at_k(self):
-        """
-        Compute average consistency in model predictions when inputs are perturbed
-        """
-        from reclist.metrics.perturbation import session_perturbation_test
-        from collections import defaultdict
-        from functools import partial
-
-        # Step 1: Generate a map from artist uri to track uri
-        substitute_mapping = defaultdict(list)
-        for track_uri, row in self.product_data.items():
-            substitute_mapping[row['artist_uri']].append(track_uri)
-
-        # Step 2: define a custom perturbation function
-        def perturb(session, sub_map):
-            last_item = session[-1]
-            last_item_artist = self.product_data[last_item['track_uri']]['artist_uri']
-            substitutes = set(sub_map.get(last_item_artist,[])) - {last_item['track_uri']}
-            if substitutes:
-                similar_item = random.sample(substitutes, k=1)
-                new_session = session[:-1] + [{"track_uri": similar_item[0]}]
-                return new_session
-            return []
-
-        # Step 3: call test
-        return session_perturbation_test(self.rec_model,
-                                         self._x_test,
-                                         self._y_preds,
-                                         partial(perturb, sub_map=substitute_mapping),
-                                         self.uri_only,
-                                         k=10)
-
-    @rec_test(test_type='shuffle_session')
-    def perturbation_shuffle_at_k(self):
-        """
-        Compute average consistency in model predictions when inputs are re-ordered
-        """
-        from reclist.metrics.perturbation import session_perturbation_test
-
-        # Step 1: define a custom perturbation function
-        def perturb(session):
-            return random.sample(session, len(session))
-
-        # Step 2: call test
-        return session_perturbation_test(self.rec_model,
-                                         self._x_test,
-                                         self._y_preds,
-                                         perturb,
-                                         self.uri_only,
-                                         k=10)
-
-    @rec_test(test_type='hits_distribution_by_slice')
-    def hits_distribution_by_slice(self):
-        """
-        Compute the distribution of hit-rate across various slices of data
-        """
-        from reclist.metrics.hits import hits_distribution_by_slice
-
-        len_map = collections.defaultdict(list)
-        for idx, playlist in enumerate(self._x_test):
-            len_map[len(playlist)].append(idx)
-        slices = collections.defaultdict(list)
-        bins = [(x * 5, (x + 1) * 5) for x in range(max(len_map) // 5 + 1)]
-        for bin_min, bin_max in bins:
-            for i in range(bin_min + 1, bin_max + 1, 1):
-                slices[f'({bin_min}, {bin_max}]'].extend(len_map[i])
-                del len_map[i]
-        assert len(len_map) == 0
-
-        return hits_distribution_by_slice(slices,
-                                          self.uri_only(self._y_test),
-                                          self.uri_only(self._y_preds),
-                                          debug=True)
-
-    @rec_test(test_type='Coverage@10')
-    def coverage_at_k(self):
-        """
-        Coverage is the proportion of all possible products which the RS
-        recommends based on a set of sessions
-        """
-        from reclist.metrics.standard_metrics import coverage_at_k
-        return coverage_at_k(self.uri_only(self._y_preds),
-                             self.product_data,
-                             # this contains all the track URIs from train and test sets
-                             k=10)
-
-    @rec_test(test_type='Popularity@10')
-    def popularity_bias_at_k(self):
-        """
-        Compute average frequency of occurrence across recommended items in training data
-        """
-        from reclist.metrics.standard_metrics import popularity_bias_at_k
-        return popularity_bias_at_k(self.uri_only(self._y_preds),
-                                    self.uri_only(self._x_train),
-                                    k=10)
-
-    @rec_test(test_type='MRR@10')
-    def mrr_at_k(self):
-        """
-        MRR calculates the mean reciprocal of the rank at which the first
-        relevant item was retrieved
-        """
-        from reclist.metrics.standard_metrics import mrr_at_k
-        return mrr_at_k(self.uri_only(self._y_preds),
-                        self.uri_only(self._y_test))
-
-    def uri_only(self, playlists: List[dict]):
-        return [[track['track_uri'] for track in playlist] for playlist in playlists]
+import json
+import os
+import time
+from abc import ABC
+from pathlib import Path
+from functools import wraps
+import pandas as pd
+from reclist.charts import CHART_TYPE
+from reclist.logs import LOGGER, logger_factory
+from reclist.metadata import METADATA_STORE, metadata_store_factory
+
+
+def rec_test(test_type: str, display_type: CHART_TYPE = None):
+    """
+    Rec test decorator
+    """
+
+    def decorator(f):
+        @wraps(f)
+        def w(*args, **kwargs):
+            return f(*args, **kwargs)
+
+        # add attributes to f
+        w.is_test = True
+        w.test_type = test_type
+        w.display_type = display_type
+        try:
+            w.test_desc = f.__doc__.lstrip().rstrip()
+        except:
+            w.test_desc = ""
+        try:
+            # python 3
+            w.name = w.__name__
+        except:
+            # python 2
+            w.name = w.__func__.func_name
+        return w
+
+    return decorator
+
+
+class RecList(ABC):
+
+    # this is the target metadata folder
+    # it can be overwritten by the user
+    # if an env variable is set
+    META_DATA_FOLDER = os.environ.get("RECLIST_META_DATA_FOLDER", ".reclist")
+
+    def __init__(
+            self,
+            model_name: str,
+            logger: LOGGER = LOGGER.LOCAL,
+            metadata_store: METADATA_STORE = METADATA_STORE.LOCAL,
+            **kwargs,
+            ):
+        """
+        :param model:
+        :param dataset:
+        """
+
+        self.name = self.__class__.__name__
+        self.model_name = model_name
+        self._rec_tests = self.get_tests()
+        self._test_results = []
+        self.logger = logger
+        self.logger_service = logger_factory(logger)(**kwargs)
+        # if s3 is used, we need to specify the bucket
+        self.metadata_bucket = kwargs["bucket"] if "bucket" in kwargs else None
+        assert self.metadata_bucket is not None if metadata_store == METADATA_STORE.S3 else True, \
+            "If using S3, you need to specify the bucket"
+        self.metadata_store_service = metadata_store_factory(metadata_store)(**kwargs)
+        self.metadata_store = metadata_store
+
+        return
+
+    def get_tests(self):
+        """
+        Helper to extract methods decorated with rec_test
+        """
+
+        nodes = {}
+        for _ in self.__dir__():
+            if not hasattr(self, _):
+                continue
+            func = getattr(self, _)
+            if hasattr(func, "is_test"):
+                nodes[func.name] = func
+
+        return nodes
+
+    def create_data_store(self):
+        """
+        Each reclist run stores artifacts in
+
+        METADATA_FOLDER/ReclistName/ModelName/RunEpochTimeMs
+        """
+        run_epoch_time_ms = round(time.time() * 1000)
+        # specify a bucket as the root of the datastore if using s3
+        bucket = self.metadata_bucket if self.metadata_store == METADATA_STORE.S3 else ''
+        # create datastore path
+        report_path = os.path.join(
+            bucket,
+            self.META_DATA_FOLDER,
+            self.name,
+            self.model_name,
+            str(run_epoch_time_ms),
+        )
+        # create subfolders in the local file system if needed
+        folders = ["artifacts", "results", "plots"]
+        if self.metadata_store == METADATA_STORE.LOCAL:
+            for folder in folders:
+                Path(os.path.join(report_path, folder)).mkdir(
+                    parents=True, exist_ok=True
+                )
+
+        return report_path
+
+    def _display_rich_table(self, table_name: str, results: list):
+        from rich.console import Console
+        from rich.table import Table
+        # build the rich table
+        table = Table(title=table_name)
+        table.add_column("Type", justify="right", style="cyan", no_wrap=True)
+        table.add_column("Description ", style="magenta", no_wrap=False)
+        table.add_column("Result", justify="right", style="green")
+        for result in results:
+            # rich needs strings to display
+            printable_result = None
+            if isinstance(result['result'], float):
+                printable_result = str(round(result['result'], 4))
+            elif isinstance(result['result'], dict):
+                printable_result = json.dumps(result['result'], indent=4)
+            elif isinstance(result['result'], list):
+                printable_result = json.dumps(
+                    result['result'][:3] + ["..."],
+                    indent=4
+                )
+            else:
+                printable_result = str(result['result'])
+            table.add_row(
+                result['name'],
+                result['description'],
+                printable_result
+                )
+        # print out the table
+        console = Console()
+        console.print(table)
+
+        return
+
+    def __call__(self, verbose=True, *args, **kwargs):
+        from rich.progress import track
+
+        self.meta_store_path = self.create_data_store()
+        # iterate through tests
+        for test_func_name, test in track(self._rec_tests.items(), description="Running RecTests"):
+            test_result = test(*args, **kwargs)
+            # we could store the results in the test function itself
+            # test.__func__.test_result = test_result
+            self._test_results.append(
+                {
+                    "name": test.test_type,
+                    "description": test.test_desc,
+                    "result": test_result,
+                    "display_type": str(test.display_type),
+                }
+            )
+            self.logger_service.write(test.test_type, test_result)
+        # finally, display all results in a table
+        self._display_rich_table(self.name, self._test_results)
+        # at the end, dump results to json and generate plots
+        test_2_fig = self._generate_report_and_plot(self._test_results, self.meta_store_path)
+        for test, fig in test_2_fig.items():
+            self.logger_service.save_plot(name=test, fig=fig)
+
+        return
+
+    def _generate_report_and_plot(self, test_results: list, meta_store_path: str):
+        """
+        Store a copy of the results into a file in the metadata store
+
+        TODO: decide what to do with artifacts
+        """
+        # dump results to json
+        report_file_name = self._dump_results_to_json(test_results, meta_store_path)
+        # generate and save plots if applicable
+        test_2_fig = self._generate_plots(test_results)
+        for test_name, fig in test_2_fig.items():
+            if self.metadata_store == METADATA_STORE.LOCAL:
+                # TODO: decide if we want to save the plot in S3 or not
+                fig.savefig(os.path.join(meta_store_path, "plots", "{}.png".format(test_name)))
+        # TODO: decide how store artifacts / if / where
+        # self.store_artifacts(report_path)
+        return test_2_fig
+
+    def _generate_plots(self, test_results: list):
+        test_2_fig = {}
+        for test_result in test_results:
+            display_type = test_result['display_type']
+            fig = None
+            if display_type == str(CHART_TYPE.SCALAR):
+                # TODO: decide how to plot scalars
+                pass
+            elif display_type == str(CHART_TYPE.BARS):
+                fig = self._bar_chart(test_result)
+            elif display_type == str(CHART_TYPE.BINS):
+                fig = self._bin_chart(test_result)
+            # append fig to the mapping
+            if fig is not None:
+                test_2_fig[test_result['name']] = fig
+
+        return test_2_fig
+
+    def _bin_chart(self, test_result: dict):
+        import matplotlib.pyplot as plt
+
+        fig, ax = plt.subplots()
+        ax.set_xlabel('x')
+        ax.set_ylabel('y')
+        ax.set_title(test_result['name'])
+        data = test_result['result']
+        assert isinstance(data, list), "data must be a list"
+        ax.hist(data, color='lightgreen', ec='black')
+
+        return fig
+
+    def _bar_chart(self, test_result: dict):
+        import matplotlib.pyplot as plt
+
+        fig, ax = plt.subplots()
+        ax.set_xlabel('x')
+        ax.set_ylabel('y')
+        ax.set_title(test_result['name'])
+        data = test_result['result'].keys()
+        ax.bar(data, [test_result['result'][_] for _ in data])
+
+        return fig
+
+
+    def _dump_results_to_json(self, test_results: list, report_path: str):
+        report = {
+            "metadata": {
+                "model_name": self.model_name,
+                "reclist": self.name,
+                "tests": list(self._rec_tests.keys()),
+            },
+            "data": test_results,
+        }
+        report_file_name = os.path.join(report_path, "results", "report.json")
+        self.metadata_store_service.write_file(
+            report_file_name,
+            report,
+            is_json=True
+        )
 
+        return report_file_name
 
-class MovieLensSimilarItemRecList(RecList):
-    @rec_test(test_type="stats")
-    def basic_stats(self):
-        """
-        Basic statistics on training, test and prediction data
-        """
-        from reclist.metrics.standard_metrics import statistics
-        return statistics(
-            self._x_train,
-            self._y_train,
-            self._x_test,
-            self._y_test,
-            self._y_preds
-        )
+    @property
+    def rec_tests(self):
+        return self._rec_tests
 
-    @rec_test(test_type='HR@10')
-    def hit_rate_at_k(self):
-        """
-        Compute the rate at which the top-k predictions contain the movie to be predicted
-        """
-        from reclist.metrics.standard_metrics import hit_rate_at_k
-        return hit_rate_at_k(
-            self.movie_only(self._y_preds),
-            self.movie_only(self._y_test),
-            k=10
-        )
 
-    @rec_test(test_type='Coverage@10')
-    def coverage_at_k(self):
-        """
-        Coverage is the proportion of all possible movies which the RS
-        recommends based on a set of movies and their respective ratings
-        """
-        from reclist.metrics.standard_metrics import coverage_at_k
-        return coverage_at_k(
-            self.movie_only(self._y_preds),
-            self.product_data,
-            k=10
-        )
 
-    @rec_test(test_type='hits_distribution')
-    def hits_distribution(self):
-        """
-        Compute the distribution of hit-rate across movie frequency in training data
-        """
-        from reclist.metrics.hits import hits_distribution
-        return hits_distribution(
-            self.movie_only(self._x_train),
-            self.movie_only(self._x_test),
-            self.movie_only(self._y_test),
-            self.movie_only(self._y_preds),
-            k=10,
-            debug=True
-        )
 
-    @rec_test(test_type="hits_distribution_by_rating")
-    def hits_distribution_by_rating(self):
-        """
-        Compute the distribution of hit-rate across movie ratings in testing data
-        """
-        from reclist.metrics.hits import hits_distribution_by_rating
-        return hits_distribution_by_rating(
-            self._y_test,
-            self._y_preds,
-            debug=True
-        )
 
-    def movie_only(self, movies):
-        return [[x["movieId"] for x in y] for y in movies]
diff --git a/reclist/recommenders/__init__.py b/reclist/recommenders/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/reclist/recommenders/prod2vec.py b/reclist/recommenders/prod2vec.py
deleted file mode 100644
index fbcdb6a..0000000
--- a/reclist/recommenders/prod2vec.py
+++ /dev/null
@@ -1,150 +0,0 @@
-import numpy as np
-from reclist.abstractions import RecModel
-from reclist.utils.train_w2v import train_embeddings
-
-
-class CoveoP2VRecModel(RecModel):
-    """
-    Implement of the prod2vec model through the standard RecModel interface.
-
-    """
-    def __init__(self, **kwargs):
-        super().__init__(**kwargs)
-
-    model_name = "prod2vec"
-
-    def train(self, products, iterations=15):
-        """
-        Takes a list of products in the COVEO dataset (with coveo dataset format) and trains a model
-
-        :param products: list of json
-        :return:
-        """
-        x_train_skus = [[e['product_sku'] for e in s] for s in products]
-        self._model = train_embeddings(x_train_skus, iterations=iterations)
-
-    def predict(self, prediction_input: list, *args, **kwargs):
-        """
-        Implement the abstract method, accepting a list of lists, each list being
-        the content of a cart: the predictions returned by the model are the top K
-        items suggested to complete the cart.
-
-        :param prediction_input:
-        :return:
-
-        """
-        predictions = []
-        for _x in prediction_input:
-            # we assume here that every X is a list of one-element, the product already in the cart
-            # i.e. our prediction_input list is [[sku_1], [sku_3], ...]
-            key_item = _x[0]['product_sku']
-            nn_products = self._model.most_similar(key_item, topn=10) if key_item in self._model else None
-            if nn_products:
-                predictions.append([{'product_sku': _[0]} for _ in nn_products])
-            else:
-                predictions.append([])
-
-        return predictions
-
-    def get_vector(self, product_sku):
-        try:
-            return list(self._model.get_vector(product_sku))
-        except Exception as e:
-            return []
-
-
-class SpotifyP2VRecModel(RecModel):
-    """
-    Implement of the prod2vec model through the standard RecModel interface.
-
-    Since init is ok, we just need to overwrite the prediction methods to get predictions
-    out of it.
-    """
-    def __init__(self, **kwargs):
-        super().__init__(**kwargs)
-
-    model_name = "prod2vec"
-
-    def train(self, playlists, iterations=15):
-        x_train_uris = [[track['track_uri'] for track in playlist] for playlist in playlists]
-        self._model = train_embeddings(x_train_uris, iterations=iterations)
-
-    def predict(self, prediction_input: list, *args, **kwargs):
-        """
-        Implement the abstract method.
-        NEP following industry best practices mentioned in https://arxiv.org/abs/2007.14906:
-        given a trained prod2vec, take all the seeded (or before-last) tracks to construct a
-        playlist vector by average pooling, and use kNN to predict the next items (or the last
-        item).
-        """
-        predictions = []
-        for _x in prediction_input:
-            embeddings = [self._model[track['track_uri']] for track in _x if track['track_uri'] in self._model]
-            if embeddings:
-                avg_playlist_vector = np.average(embeddings, axis=0)
-                nn_tracks = self._model.similar_by_vector(avg_playlist_vector, topn=100)
-                predictions.append([{'track_uri':_[0]} for _ in nn_tracks])
-            else:
-                predictions.append([])
-
-        return predictions
-
-    def get_vector(self, track_uri):
-        try:
-            return list(self._model.get_vector(track_uri))
-        except Exception as e:
-            return []
-
-class MovieLensP2VRecModel(RecModel):
-    """
-    Prod2Vec implementation for MovieLens 25M dataset
-    """
-    model_name = "prod2vec"
-
-    def __init__(self, **kwargs):
-        super().__init__(**kwargs)
-
-    def train(self, movies, iterations=15):
-        # Get the movie ID and rating for each movie for each unique user
-        x_train = [[(x["movieId"], x["rating"]) for x in y] for y in movies]
-        self._model = train_embeddings(x_train, iterations=iterations)
-
-    def predict(self, prediction_input, *args, **kwargs):
-        """
-        Predicts the top 10 similar items recommended for each user according
-        to the movies that they've watched and the ratings that they've given
-
-        :param prediction_input: a list of lists containing a dictionary for
-                                 each movie watched by that user
-        :return:
-        """
-        all_predictions = []
-        for movies in prediction_input:
-            predictions = []
-            emb_vecs = []
-            for movie in movies:
-                emb_vec = self.get_vector(movie)
-                if emb_vec:
-                    emb_vecs.append(emb_vec)
-            if emb_vecs:
-                # Calculate the average of all the latent vectors representing
-                # the movies watched by the user as is done in https://arxiv.org/abs/2007.14906
-                avg_emb_vec = np.mean(emb_vecs, axis=0)
-                nn_products = self.model.similar_by_vector(avg_emb_vec, topn=10)
-                for elem in nn_products:
-                    predictions.append({"movieId": elem[0][0], "rating": elem[0][1]})
-            all_predictions.append(predictions)
-        return all_predictions
-
-    def get_vector(self, x):
-        """
-        Returns the latent vector that corresponds to the movie ID and the rating
-
-        :param x:
-        :return:
-        """
-        movie = (x["movieId"], x["rating"])
-        try:
-            return list(self.model.get_vector(movie))
-        except Exception as e:
-            return []
diff --git a/reclist/similarity_models.py b/reclist/similarity_models.py
new file mode 100644
index 0000000..bc09257
--- /dev/null
+++ b/reclist/similarity_models.py
@@ -0,0 +1,140 @@
+from abc import ABC, abstractmethod
+import json
+import os
+
+class SimilarityModel(ABC):
+
+    def __init__(self, *args, **kwargs):
+        pass
+
+    @abstractmethod
+    def similarity_binary(self, query, target, *args, **kwargs):
+        pass
+
+    @abstractmethod
+    def similarity_gradient(self, query, target, *args, **kwargs):
+        pass
+
+
+class FakeSimilarityModel(SimilarityModel):
+
+    """
+    TODO: this is a stub, to showcase the idea of a similarity model
+    and how it can be implemented in different ways
+    """
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.similarity_threshold = kwargs.get("similarity_threshold", 0.5)
+        return
+
+    def similarity_binary(self, query, target, *args, **kwargs) -> bool:
+        """
+        We can binarize the similarity by taking the threshold over
+        the distance in the latent space
+        """
+        similarity_score = self.similarity_gradient(query, target, *args, **kwargs)
+        return similarity_score > self.similarity_threshold
+
+    def similarity_gradient(self, query, target, *args, **kwargs) -> float:
+        """
+        We return the distance in the latent space
+        """
+        # TODO: replace this with the actual distance
+        from random import random
+        return random()
+
+
+class GPT3SimilarityModel(SimilarityModel):
+
+    API_URL = 'https://api.openai.com/v1/completions'
+    MODEL = "text-davinci-002"
+    TEMPERATURE = 0
+    SIMILARITY_PROMPT = '''
+        Imagine you are shopping for fashion products in a fashion store.
+        Your best friend tells you to buy something as close as possible to this item:
+
+        {}.
+
+        The shopping assistant proposes the following alternative:
+
+        {}.
+
+        Is this second product similar enough to the one suggested by your friend? Provide a yes/no answer.
+    '''
+
+    """
+    TODO: this is a stub, to showcase the idea of a similarity model
+    and how it can be implemented in different ways
+    """
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        # get the key from kwargs or fall back on envs
+        self.api_key = kwargs.get("api_key", os.environ["OPENAI_API_KEY"])
+        # you can override the model and temperature
+        self.MODEL = kwargs.get("model", self.MODEL)
+        self.TEMPERATURE = kwargs.get("temperature", self.TEMPERATURE)
+        self.SIMILARITY_PROMPT = kwargs.get("similarity_prompt", self.SIMILARITY_PROMPT)
+        return
+
+    def similarity_binary(self, query, target, *args, **kwargs) -> bool:
+        import requests
+        # serialize the query and target
+        query_str = ''
+        for k, v in query.items():
+            query_str += '{}: {} | '.format(k.strip(), v.replace('\n', ' ').strip())
+        target_str = ''
+        for k, v in target.items():
+            target_str += '{}: {} | '.format(k.strip(), v.replace('\n', ' ').strip())
+        final_prompt = self.SIMILARITY_PROMPT.format(query_str, target_str).strip()
+        data =  {
+                "model": self.MODEL,
+                "prompt": final_prompt,
+                "temperature": self.TEMPERATURE,
+                "max_tokens": 10
+                }
+        headers = {
+            'content-type': 'application/json',
+            "Authorization": "Bearer {}".format(self.api_key)
+            }
+        r = requests.post(self.API_URL, data=json.dumps(data), headers=headers)
+        # NOTE: we break if API call fails, alternative behavior are possible of course
+        if r.status_code != 200:
+            raise Exception("Error in GPT3 API call: {}".format(r.text))
+
+        completion = json.loads(r.text)['choices'][0]['text']
+        if kwargs.get("verbose", False):
+            print("Query: {}".format(query_str))
+            print("Target: {}".format(target_str))
+            print("Prompt: {}".format(final_prompt))
+            print("Completion: {}".format(completion))
+        # if the first word is Yes, we return True
+        first_word = completion.strip().split(" ")[0].lower()
+
+        return first_word == "yes"
+
+    def similarity_gradient(self, query, target, *args, **kwargs) -> float:
+        raise Exception("No gradient is available for GPT3")
+
+# test the similarity model with open ai :clownface:
+#sim_model = GPT3SimilarityModel(api_key=os.environ["OPENAI_API_KEY"])
+#p1 = {
+#    "name": "logo-print cotton cap",
+#    "brand": 'Palm Angels',
+#    "description": '''
+#    Known for a laid-back aesthetic, Palm Angels knows how to portray its Californian inspiration. This classic cap carries the brand's logo printed on the front, adding a touch of recognition to a relaxed look.
+#    '''
+#}
+#p2 = {
+#    "name": "monogram badge cap",
+#    "brand": 'Balmain',
+#    "description": '''
+#    Blue cotton monogram badge cap from Balmain featuring logo patch to the front, mesh detailing, fabric-covered button at the crown and adjustable fit.
+#    '''
+#}
+#similarity_judgement = sim_model.similarity_binary(p1, p2, verbose=False)
+#print("P1 {} and P2 {} are similar: {}".format(p1["name"], p2["name"], similarity_judgement))
+
+
+
diff --git a/reclist/utils/__init__.py b/reclist/utils/__init__.py
deleted file mode 100644
index e69de29..0000000
diff --git a/reclist/utils/config.py b/reclist/utils/config.py
deleted file mode 100644
index d7d8fe6..0000000
--- a/reclist/utils/config.py
+++ /dev/null
@@ -1,48 +0,0 @@
-from appdirs import *
-from pathlib import Path
-import requests
-from tqdm import tqdm
-from enum import Enum
-
-COVEO_INTERACTION_DATASET_S3_URL = 'https://reclist-datasets-6d3c836d-6djh887d.s3.us-west-2.amazonaws.com/coveo_sigir.zip'
-SPOTIFY_PLAYLIST_DATASET_S3_URL = 'https://reclist-datasets-6d3c836d-6djh887d.s3.us-west-2.amazonaws.com/small_spotify_playlist.zip'
-MOVIELENS_DATASET_S3_URL = "https://reclist-datasets-6d3c836d-6djh887d.s3.us-west-2.amazonaws.com/movielens_25m.zip"
-
-
-def download_with_progress(url, destination):
-    """
-    Downloads a file with a progress bar
-
-    :param url: url from which to download from
-    :destination: file path for saving data
-    """
-    try:
-        response = requests.get(url, stream=True)
-        response.raise_for_status()
-    except requests.exceptions.RequestException as e:
-        raise SystemExit(e)
-    with tqdm.wrapattr(open(destination, "wb"), "write",
-                       miniters=1, desc=url.split('/')[-1],
-                       total=int(response.headers.get('content-length', 0))) as fout:
-        for chunk in response.iter_content(chunk_size=4096):
-            fout.write(chunk)
-
-
-def get_cache_directory():
-    """
-    Returns the cache directory on the system
-    """
-    appname = "reclist"
-    appauthor = "reclist"
-    cache_dir = user_cache_dir(appname, appauthor)
-
-    Path(cache_dir).mkdir(parents=True, exist_ok=True)
-
-    return cache_dir
-
-
-class Dataset(Enum):
-    COVEO = 'coveo'
-    COVEO_INTERNAL = 'coveo-internal'
-    MOVIELENS = 'movielens'
-    SPOTIFY = 'spotify'
diff --git a/reclist/utils/train_w2v.py b/reclist/utils/train_w2v.py
deleted file mode 100644
index f59ab54..0000000
--- a/reclist/utils/train_w2v.py
+++ /dev/null
@@ -1,35 +0,0 @@
-import gensim
-
-
-def train_embeddings(
-    sessions: list,
-    min_c: int = 3,
-    size: int = 48,
-    window: int = 5,
-    iterations: int = 15,
-    ns_exponent: float = 0.75,
-    is_debug: bool = True):
-    """
-    Train CBOW to get product embeddings with sensible defaults (https://arxiv.org/abs/2007.14906).
-
-    :param sessions: list of lists, as user sessions are list of interactions
-    :param min_c: minimum frequency of an event for it to be calculated for product embeddings
-    :param size: output dimension
-    :param window: window parameter for gensim word2vec
-    :param iterations: number of training iterations
-    :param ns_exponent: ns_exponent parameter for gensim word2vec
-    :param is_debug: if true, be more verbose when training
-
-    :return: trained product embedding model
-    """
-    model = gensim.models.Word2Vec(sentences=sessions,
-                                   min_count=min_c,
-                                   vector_size=size,
-                                   window=window,
-                                   epochs=iterations,
-                                   ns_exponent=ns_exponent)
-
-    if is_debug:
-        print("# items in the space: {}".format(len(model.wv.index_to_key)))
-
-    return model.wv
diff --git a/requirements.txt b/requirements.txt
index f33a61c..1e9a1bb 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,4 +1,5 @@
-gensim==4.2.0
+pandas>=1.3.5
+gensim==4.3.1
 appdirs==1.4.4
 wget==3.2
 requests>=2.22.0
diff --git a/setup.py b/setup.py
index 9acf4b6..2a4de9a 100644
--- a/setup.py
+++ b/setup.py
@@ -2,47 +2,62 @@
 
 """The setup script."""
 
-from setuptools import setup, find_packages
+from setuptools import find_packages, setup
 
-with open('README.rst') as readme_file:
+with open("README.rst") as readme_file:
     readme = readme_file.read()
 
-with open('HISTORY.rst') as history_file:
+with open("HISTORY.rst") as history_file:
     history = history_file.read()
 
-with open('requirements.txt') as f:
+with open("requirements.txt") as f:
     requirements = f.read().splitlines()
 
-setup_requirements = ['pytest-runner', ]
+setup_requirements = [
+    "pytest-runner",
+]
 
-test_requirements = ['pytest>=3', ]
+test_requirements = [
+    "pytest>=3",
+]
+
+style_packages = [
+    "black==22.3.0",
+    # "flake8==5.0.4",
+    "isort==5.10.1",
+]
 
 setup(
+
     author="RecList",
     author_email='',
     python_requires='>=3.6',
     classifiers=[
-        'Development Status :: 2 - Pre-Alpha',
-        'Intended Audience :: Developers',
-        'License :: OSI Approved :: MIT License',
-        'Natural Language :: English',
-        'Programming Language :: Python :: 3',
-        'Programming Language :: Python :: 3.6',
-        'Programming Language :: Python :: 3.7',
-        'Programming Language :: Python :: 3.8',
+        "Development Status :: 2 - Pre-Alpha",
+        "Intended Audience :: Developers",
+        "License :: OSI Approved :: MIT License",
+        "Natural Language :: English",
+        "Programming Language :: Python :: 3",
+        "Programming Language :: Python :: 3.6",
+        "Programming Language :: Python :: 3.7",
+        "Programming Language :: Python :: 3.8",
     ],
     description="RecList",
     install_requires=requirements,
     license="MIT license",
-    long_description=readme + '\n\n' + history,
-    long_description_content_type='text/x-rst',
+    long_description=readme + "\n\n" + history,
+    long_description_content_type="text/x-rst",
     include_package_data=True,
-    keywords='reclist',
-    name='reclist',
-    packages=find_packages(include=['reclist', 'reclist.*']),
-    test_suite='tests',
+    keywords="reclist",
+    name="reclist",
+    packages=find_packages(include=["reclist", "reclist.*"]),
+    test_suite="tests",
     tests_require=test_requirements,
+
     url='https://github.com/jacopotagliabue/reclist',
     version='0.3.1',
     zip_safe=False,
+    extras_require={
+        "dev": style_packages + ["pre-commit==2.20.0"],
+    },
 )


Test patch:
diff --git a/tests/test_reclist.py b/tests/test_reclist.py
index 19d53bd..ed60ce6 100644
--- a/tests/test_reclist.py
+++ b/tests/test_reclist.py
@@ -2,48 +2,48 @@
 
 """Tests for `reclist` package."""
 
-from reclist.datasets import *
-from reclist.metrics.standard_metrics import mrr_at_k
-from reclist.datasets import CoveoDataset, SpotifyDataset, MovieLensDataset
-from reclist.recommenders.prod2vec import CoveoP2VRecModel, SpotifyP2VRecModel, MovieLensP2VRecModel
-from reclist.reclist import CoveoCartRecList, SpotifySessionRecList, MovieLensSimilarItemRecList
 import random
 
-def test_basic_dataset_downloading():
-    CoveoDataset()
-    MovieLensDataset()
-    MovieLensDataset()
+import pandas as pd
+from reclist.metrics.standard_metrics import hit_rate_at_k, mrr_at_k
 
 
-def test_coveo_example():
-    # get the coveo data challenge dataset as a RecDataset object
-    coveo_dataset = CoveoDataset()
 
-    # re-use a skip-gram model from reclist to train a latent product space, to be used
-    # (through knn) to build a recommender
-    model = CoveoP2VRecModel()
-    x_train = random.sample(coveo_dataset.x_train, 2000)
-    model.train(x_train, iterations=1)
 
-    # instantiate rec_list object, prepared with standard quantitative tests
-    # and sensible behavioral tests (check the paper for details!)
-    rec_list = CoveoCartRecList(
-        model=model,
-        dataset=coveo_dataset
+def test_hits():
+
+    df_b = pd.DataFrame(
+        [[9, 0, 32, 12], [1, 0, 7, 12], [0, 1, 5, 12], [12, 32, 66, 99]]
+    )
+    df_a = pd.DataFrame([[0], [1], [5], [99]])
+    df_c = pd.DataFrame(
+        [
+            [10000, 10000, 10000],
+            [10000, 10000, 10000],
+            [10000, 10000, 10000],
+            [10000, 10000, 10000],
+        ]
     )
 
-    # invoke rec_list to run tests
-    rec_list(verbose=True)
+    assert hit_rate_at_k(df_b, df_a, 5) == 1
+    assert hit_rate_at_k(df_b, df_a, 100) == 1  # out of bounds
+    assert hit_rate_at_k(df_b, df_a, 1) == 0.25  # 1 out of 4
+    assert hit_rate_at_k(df_b, df_a, 2) == 0.5  # 2 out of 4
+    assert hit_rate_at_k(df_b, df_a, 3) == 0.75  # 3 out of 4
+
+    assert hit_rate_at_k(df_c, df_a, 3) == 0
+    assert hit_rate_at_k(df_c, df_a, 100) == 0
+    assert hit_rate_at_k(df_c, df_a, 5) == 0
 
 
 def test_mrr():
     """
     Testing MRR works as intended
     """
-    list_a = [[0], [1]]
-    list_b = [[0, 1], [1, 0]]
-    list_c = [[2, 3], [0, 1]]
+    df_a = pd.DataFrame([[0], [1]])
+    df_b = pd.DataFrame([[0, 1], [1, 0]])
+    df_c = pd.DataFrame([[2, 3], [0, 1]])
 
-    assert mrr_at_k(list_b, list_a, 1) == 1
-    assert mrr_at_k(list_c, list_a, 21) == 0.25
-    assert mrr_at_k(list_c, list_a, 1) == 0
+    assert mrr_at_k(df_b, df_a, 1) == 1
+    assert mrr_at_k(df_c, df_a, 2) == 0.25
+    assert mrr_at_k(df_c, df_a, 1) == 0