How to Generate Professional API Docs in Minutes from Docstrings | by Tirthajyoti Sarkar | Towards Data Science

We all love good and comprehensive documentation when we use a new library (or re-use our favorite one for the millionth time), don’t we?
Imagine how would you feel if they took away all the docs from Scikit-learn or TensorFlow website. You will feel pretty powerless, wouldn’t you?
Documentation is important. High-quality documentation (particularly for an open-source project) demonstrates,
Care for the project
Care for the end-users or customers(your code is read many more times than it is written)
A productive bent of mind so that the developer can utilize the code base more efficiently for debugging and reference or users can generate bugs/issues easily (what is productive data science)
A tendency to take technical projects to a logical conclusion i.e., not leave one’s hobby project at the Jupyter notebook level, but make it reusable and distribution-worthy for a larger user base (you distinguish yourself from other data scientists this way)
Here is an article with some excellent examples...
nordicapis.com
But, writing documentation often feels like a chore. It’s an additional burden on top of the cognitive load that is experienced by every developer while working on a large-scale project.
What if we had a tool that can generate professional-looking documentation i.e. ready to be pushed to a docs server or website for public consumption — right from the raw Python functions that we write anyway?
In this article, we introduce such a nifty little tool and show some basic examples.
We all write a multitude of Python functions and create a host of Python classes for our data science projects and open-source packages, don’t we? In fact, even for causal analysis and hobby projects, we should build this habit of object-oriented programming style into our codebase.
It would be pretty neat to be able to generate good-looking API documentation right from those function and class definitions without writing a single extra line of HTML/CSS code.
Just pip install the package,
pip install pdoc3
NOTE: There is an older package called pdoc. You can use that too for documentation building, it’s a fork of this project. But here we are using the more recent framework.
Start with a simple add function. Suppose it is contained in a file called math-func.py in your local workspace.
def addition(num1, num2=0):    """    Adds two numbers        Args:        num1: The first number        num2: The second number, default 0        Returns:        The result of the addition process    """        return (num1+num2)
So, the actual code is only a single return statement but the docstring is long.
That’s the key. The docstring.
The docstring is at the heart of how pdoc works. From their website, it is the “command-line application can be used to render a project’s documentation as static HTML files. It also includes a live-reloading webserver to preview changes.”
So, the flow above changes to,
And, how do you generate that web preview? Just one CLI command.
pdoc --http localhost:8080 math-func.py
It starts a live web server at the localhost:8080 port. When you open that page in your browser,
Starting pdoc server on localhost:8080pdoc server ready at http://localhost:8080
Your browser shows a page like the following. First, you will see the page with two links, and when you click on the math-func link, you will see the API.
Cool, isn’t it? But it looks pretty basic. Let’s see if we can add little bells and whistles to it.
I am a big fan of the markdown format for writing documentation. Don’t you use it often too for your Github Readme docs? The great thing about pdoc is that it allows seamless integration of markdown text inside your docstring.
Suppose, we add a second function mult to our math-func.py module,
def mult(num1, num2=1):    """    Multiplies two numbers        Args:        `num1`: The first number            `num2`: The second number, default `1`        Returns:        The result of the **multiplication** process    """        return (num1*num2)
Note the slight changes to the docstring with markdown symbols like the backticks `...` and ** ... ** for boldface.
Now, if you just save this file, the webpage is automatically reloaded with your changes.
Notice how the arguments num1 and num2 are rendered as inline code blocks and how the word multiplication is in boldface in your API docs. That’s markdown magic, right inside from your docstring.
You can even show a code block inside your docstring with the three backticks markdown. The resulting API (for our next divide function),
All we had to do is to put this in the docstring,
"""...Handles exception by a check,        ```python    if num2 != 0:        return (num1/num2)    else:        raise ValueError('The second argument cannot be zero')    ```     """
Note the use of the identifier python after triple backticks to render the code block in a nicely formatted manner in the API docs.
You can have basic LaTeX math expressions inside your docstring and render them beautifully on the API doc. For example, we write a Pythagorian distance calculation function with the following docstring,
def pythagorus(num1, num2=0):    """    ### Description:        Calculates the root-sum-of-square of two numbers        ### Args:        `num1`: The first number            `num2`: The second number, default `0`        ### Returns:        The result of the **pythagorus** formula        $$dist = \\sqrt { a^2+b^2 }$$    """
We need to do the following two things,
Use $$ ... $$ to wrap a LaTeX expression
Use two backslash characters to denote the usual backslash inside a LaTeX expression. This is to handle the usual role of backslash as the escape character inside a Python docstring.
And, the result is,
Also note the index of functions that we are building on the left side of the page. This is like a Table of Contents and transports you anywhere in the document when you click on those links to the corresponding function. This is built module by module. Here our main module is, of course, the math-func.py file.
As you might have noticed, the index is alphabetically sorted (by function names). Although in our source code, we added the divide function later than the mult function, it is indexed in the second place.
We covered basic usage and options in the previous section. Here, we want to show a few more global options and utilities that can be used for more control and usability.
You can generate either,
HTML pages: With pdoc --html <filename.py> . This puts the module HTML files in a ./html folder inside your working directory. You can have an optional directory flag to put the HTML files inside that.
Server: With pdoc --http HOST:PORT <filename.py> to show the live preview of the API docs on a webpage with a user-defined HOST and PORT.
You can build documentation with standard class objects just like functions. For example, we turn the entire math-func code into a class and also add a second Dataframe class to our module. When we generate the documentation for that, it looks like,
If you put your classes and functions in a single Python script (module), then it will show up like that,
Fundamentally, pdoc3 is a Python package, and therefore you can use it in your Python code to generate documentation in a programmatic manner. Here is a boilerplate code example,
import pdocmodules = ['a', 'b']  # Public submodules are auto-importedcontext = pdoc.Context()modules = [pdoc.Module(mod, context=context)           for mod in modules]pdoc.link_inheritance(context)def recursive_htmls(mod):    yield mod.name, mod.html()    for submod in mod.submodules():        yield from recursive_htmls(submod)for mod in modules:    for module_name, html in recursive_htmls(mod):        ...  # Process
Note that mod.html() is essentially a function that returns the raw HTML string after processing the module’s docstrings. This is what is rendered using the pdoc --html <filename.py> command. In the code above, you retrieve it directly using your code and can manipulate it further anyway.
You can use pdoc3 to build a complete module in one shot. For that, you just have to put the necessary files in the usual module/sub-module hierarchy like a standard Python package.
For example, we may have this simple directory structure (note the __init__.py as required by Python namespace) of the directory math-mod inside which we have put these files in sub-directories.
We can run one-line command,
pdoc --http localhost:8080 -c latex_math=True math-mod
We will get the following structure,
Building high-quality documentation can feel like an additional burden on top of the programming and technical problem-solving part of a software project. Data science is no exception. However, great data science and machine learning projects and frameworks are truly great because of how easy they are to use by anybody due to their extensive API documentation.
In this article, we showed how to use a nifty little Python package to create beautiful API documentation (complete with markdown syntax and LaTeX math rendering) right from the comfort of function docstrings.
Hopefully, this breaks down the barrier of producing good documentation for the next open-source project you are working on.
Loved the article? Become a Medium member to continue learning without limits. I’ll receive a portion of your membership fee if you use the following link, with no extra cost to you.