Automate your Model Documentation using H2O AutoDoc
By: Parul Pandey
Tags: Documentation, Scikit Learn, Machine Learning, Regulatory Compliance

Proper documentation is essential not only for your own credibility but the credibility of the organization you represent.

The Federal Reserve’s 2011 guidelines state that model risk assessment and management would be ineffective without adequate documentation. A similar requirement is put forward today by many regulatory and corporate governance bodies. This clearly means that model documentation today is more of a necessity than a choice. However, there is still no denying that it is one of the most time-consuming jobs for a data scientist. As opposed to building and validating machine learning models, describing how a model works in detail is tedious and takes a considerable amount of time and effort, let alone dealing with the issues of consistency, clarity, and collaboration.

What if there was a way to automate the entire documentation process? Well, this is precisely the issue that the H2O AutoDoc tries to address by creating comprehensive and high-quality model documentation in minutes. The AutoDoc feature frees up the user from the time-consuming task of documenting and summarizing their workflow while building machine learning models. Additionally, it increases the consistency of documentation by applying a standard template across all models which in a way is essential for model governance, reproducibility, and regulatory compliance.

Challenges in creating a robust documentation

Typical model documentation consists of the following attributes — how a model was created, the training and the test data characteristics, what alternatives were considered, model evaluation strategies, information on model performance, etc. Documenting the models is necessary both from a best practice point of view as well as a vital requirement from the business point of view. The figure below highlights the major challenges associated with documentation in the machine learning domain.

Challenges associating with manually documenting models | Image by Author

But creating good documentation isn’t a piece of cake, and at times, many teams struggle with it. The process is often tedious and time-consuming for the business because the data scientist could be using that time to build additional models and create more value. Additionally, inconsistent or inaccurate model documentation can be an issue for model validation, governance, and regulatory compliance.

A better idea: Automate the documentation process

H2O AutoDoc

Documentation template | Image by Author

Automated Model Documentation (H2O AutoDoc) is a new time-saving ML documentation product from H2O.ai. H2O AutoDoc can automatically generate model Documentation for supervised learning models created in H2O-3 and Scikit-Learn. Interestingly, automated documentation is already being used in production as part of H2O Driverless AI — the AutoML platform. The good news is that this industry-leading capability is now available as a new standalone module.

Key Features of H2O AutoDoc

Inherently, H2O AutoDoc is a Python package for creating automatic reports for supervised learning models. It comes equipped with the following features:

Distributed automatic document generation in Microsoft Word (.docx) and Markdown (.md) formats.

and formats. Out-of-the-box documentation template included

Customizable templates to fit unique business needs, internal best practices, and compliance requirements

Support for a variety of supervised models generated in H2O-3 and Scikit-Learn

Advantages of using H2O AutoDoc

H2O AutoDoc provides various advantages over the traditional method of manual documentation.

Advantages of using H2O AutoDoc | Image by Author