Navigatable breadcrumbs and technical debts
Tags: haskell, dhall, gsoc
Published: 2020-07-09 00:25:04

# TL-DR

If you want to see my current progress, check out here:

https://hydra.dhall-lang.org/build/64022/download/1/docs/


# dhall-docs 

Hi there again! In this new post, I'm going to explain about two new features on `dhall-docs`:

* Cleaner Haskell API;
* Navigatable breadcrumbs;
* Test-setup

## Cleaner Haskell API

Previously, all the functions returned some `IO` wrapped value, and this makes tests complicated to setup.

The first thing I did before setup tests was to split all core logic behind docs generation from the `IO` related tasks.

Posting the whole code here is a non-sense (~436 LOC), but the API for dhall-docs package ended up like this in summary:

```haskell
-- | The result of the doc-generator pure component
data GeneratedDocs a = GeneratedDocs [DocsGenWarning] a

{- `GeneratedDocs` is an instance of MonadWriter to make warning logging easier,
   omitted the instance declaration since it makes some noise in the post -}

{-| Generate all of the docs for a package. This function does all the `IO ()`
    related tasks to call `generateDocsPure`
-}
generateDocs
    :: Path Abs Dir -- ^ Input directory
    -> Path Abs Dir -- ^ Link to be created to the generated documentation
    -> Text         -- ^ Package name, used in some HTML titles
    -> IO ()

{-| Generates all the documentation of dhall package in a pure way i.e.
    without an `IO` context. This lets you generate documentation from a list of
    dhall-files without saving them to the filesystem.

    If you want the `IO` version of this function, check `generateDocs`
-}
generateDocsPure
    :: Text                    -- ^ Package name
    -> [(Path Rel File, Text)] -- ^ (Input file, contents)
    -> GeneratedDocs [(Path Rel File, Text)]
```

This introduced some cool benefits:

* Pure code is easier to test.
* From my little Haskell experience, the less you are coupled with an `IO` context the better.
* The code is cleaner than before.

Thanks to my mentors for pushing me into this direction!

## Breadcrumbs generation

The challenging part on breadcrumbs generation is to wire up the links to other super-folders correctly since they depend on the type of the HTML file you're generating.

Let's take the following breadcrumb:

```
my / super / breadcrumb
```

The last breadcrumb component is never a link since it always refers to the current page you're looking at. If you're on an index page, the page where you'll be redirected will be something like this:

```
# "breadcrumb" is your current page
my                 / super           / breadcrumb
"../../index.html"   "../index.html"   no link 
```

whereas if you were on a Dhall file page, it will render the link like this:

```
# "breadcrumb" is your current page
my                 / super           / breadcrumb     / file.dhall
"../../index.html"   "../index.html"   "./index.html"   no-link
```

Get this right was cumbersome by using only lists, so I created my custom list-like ADT:

```haskell
-- | ADT for handling bread crumbs. This is essentially a backward list
data Breadcrumb
    = Crumb Breadcrumb String
    | EmptyCrumb
    deriving Show

{-| Convert a relative path to a `Breadcrumb`.

>>> relPathToBreadcrumb [reldir|a/b/c|]
Crumb (Crumb (Crumb EmptyCrumb "a") "b") "c"
>>> relPathToBreadcrumb [reldir|.|]
Crumb EmptyCrumb ""
>>> relPathToBreadcrumb [relfile|c/foo.baz|]
Crumb (Crumb EmptyCrumb "c") "foo.baz"
-}
relPathToBreadcrumb :: Path Rel a -> Breadcrumb
```

The reason why it's a backward list is to make it easier to generate the relative links backward while mapping these breadcrumbs to `Html ()`.

After adding this, we started noticing that the code was growing a lot and things start to complicate. We needed a way to ensure that adding a new feature doesn't break anything else. Oh yeah, we needed tests!

(before digging into the test-setup section, I'll show you some screenshot of how the breadcrumbs look)

On a Dhall's file documentation
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/1yil2let2yekf3m47kec.png)

On an index:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/bx4qvrdycahku362514d.png)

## Test-setup

Currently, our Haskell API changes frequently on each PR, and testing each function in isolation will complicate things a little.

Remember that the input of `dhall-docs` is a list of dhall files with their relative paths, and the output is a list of generated HTML documents that represents the generated documentation.

We want out test-setup to be easy to maintain and that adding new test cases is as easy as adding an input file and an _expected_ output file

A golden-tests setup is really good for this! Golden tests are structured like this

* There are two types of files:
  * Input files: The input of the function
  * Output files: Are written by the function given an input file
  * Golden files: The expected output of the function. The test-runner should compare them with the output files after execution.

You may be wondering that this is no different as the usual test-setup like:

```
actual <- function(input)
shouldBeEqual(actual, expected)
```

but when we are dealing with tests that involve accessing the file-system, this is a better approach.

Almost all tests-setup in `dhall-haskell` are written in `tasty`, which has several sub-packages that can be seen as _plugins_. For golden-tests, we decided to use `tasty-silver`.

The core of our test-setup is in the `goldenVsAction` function from the `tasty-silver` function:

```haskell
goldenVsAction
  :: TestName -- ^ test name
  -> FilePath -- ^ path to the «golden» file (the file that contains correct output)
  -> IO a -- ^ action that returns a text-like value.
  -> (a -> T.Text) -- ^ Converts a value to it's textual representation.
  -> TestTree -- ^ the test verifies that the returned textual representation
```

Look that we don't use output files because they mess a little with the VCS. In this function, we mimic writing a file via _actions_.

The test-runner includes a `--accept` flag that updates or creates golden files in case of test-failures. This is handy to be used when we add a new feature that changes all outputs, and we want to update them in a single run; or when we add a new test-file to generate its corresponding golden file.

The whole test-setup fits under 66 lines of code, and we probably won't have the need to edit it again in the future: If we want to write a test for a new feature or bug, we add the file in the test package directory.

## Before leaving

Thanks for reading! Keep in tune for my next posts!