npm package: @ndcode/disk_build
Description: Helper function for checking and rebuilding disk objects.
Author: Nick Downing
Latest version: 0.1.1

# Disk Build helper

An NDCODE project.

## Overview

The `disk_build` package exports a function
`disk_build(pathname, build_func, diag)`,
which given a pathname to an existing source file, will generate a pathname for
a corresponding on-disk output file, check whether this exists and is newer
than the original, and if not, rebuild it via a caller-provided function.

## Calling API

The interface for the `disk_build`-provided helper function `disk_build()` is:

`await disk_build(pathname, build_func, diag)` &mdash; checks if the source
file given in `pathname` exists, generates the corresponding output pathname
and dependency-file pathname, reads the dependency file if possible, checks the
output is up-to-date with respect to all dependencies, and if not, generates
a temporary output pathname `temp_pathname`, calls the user-provided callback
function `build_func(temp_pathname)` to build the output into the temporary,
possibly writes a new dependency file, then renames everything into place.

Finally, it returns an dictionary `{deps: ..., pathname: ...}` where `deps` is
a list of dependencies not including the original, and `pathname` is the output
file; these might come from the existing disk files or newly written ones.

The interface for the user-provided callback function `build_func()` is:

`await build_func(temp_pathname)` &mdash; the user must create the file
`temp_pathname` and then return, or alternatively an exception can be thrown.
The return value can be either `undefined` if no dependency file should be
written, or a list of pathnames referred to (not including the original).

## About asynchronicity

There is nothing too complicated here, both `disk_build()` and `build_func()`
are asynchronous and thus return a `Promise`, and when the inner `Promise`
resolves, the outer `Promise` resolves in turn. Either level throws exceptions,
with various causes such as disk errors, file not found, syntax error, etc.

There is no protection against multiple clients trying to access the same file
at once while the asychronous building operation proceeds. This is because the
accesses to `disk_build()` are supposed to be wrapped in a `BuildCache`, which
will exclude multiple access. There was no point making `disk_build()` keep a
dictionary of operations in progress when `BuildCache` already does this, since
`disk_build()` does not otherwise need any state kept in RAM (it has the disk).

## Usage example

Simple usage, showing use of the `clean-css` CSS minimizer with `disk_build()`,
in such a way as to return a string containing the minified CSS from the given
input file described by `pathname`, without re-converting if already converted:
```
let CleanCSS = require('clean-css')
let disk_build = require('@ndcode/disk_build')
let fs = require('fs')
let util = require('util')

let clean_css = new CleanCSS({returnPromise: true})
let fs_readFile = util.promisify(fs.readFile)
let fs_writeFile = util.promisify(fs.writeFile)

let get_css_min = async pathname => {
  let render = await disk_build(
    pathname,
    async temp_pathname => {
      let render = await clean_css.minify(
        await fs_readFile(pathname, {encoding: 'utf-8'})
      )
      return /*await*/ fs_writeFile(
        temp_pathname,
        render.styles,
        {encoding: 'utf-8'}
      )
    },
    true // diagnostics on
  )
  return /*await*/ fs_readFile(render.pathname)
}
```
Since it is rather common that various minimizers and template renderers return
an dictionary containing the result of processing plus some other information,
and so does `disk_cache()`, we've adopted the convention of putting the result
of such calls in a variable called `render`, despite multiple use of the name.

A more complicated example using a `BuildCache` to exclude multiple callers:
```
let BuildCache = require('@ndcode/build_cache')
let CleanCSS = require('clean-css')
let disk_build = require('@ndcode/disk_build')
let fs = require('fs')
let util = require('util')

let build_cache = new BuildCache(true) // diagnostics on
let clean_css = new CleanCSS({returnPromise: true})
let fs_readFile = util.promisify(fs.readFile)
let fs_writeFile = util.promisify(fs.writeFile)

let get_css_min = pathname => /*await*/ build_cache.get(
  pathname,
  async result => {
    let render = await disk_build(
      pathname,
      async temp_pathname => {
        let render = await clean_css.minify(
          await fs_readFile(pathname, {encoding: 'utf-8'})
        )
        return /*await*/ fs_writeFile(
          temp_pathname,
          render.styles,
          {encoding: 'utf-8'}
        )
      },
      true // diagnostics on
    )
    result.value = /*await*/ fs_readFile(render.pathname)
  }
}
```
In the above there was no dependency tracking needed or wanted, since CSS files
cannot include further CSS source files. `Less` files can, handled as follows:
```
let BuildCache = require('@ndcode/build_cache')
let disk_build = require('@ndcode/disk_build')
let fs = require('fs')
let util = require('util')
let less = require('less/lib/less-node')
let path = require('path')

let build_cache = new BuildCache(true) // diagnostics on
let fs_readFile = util.promisify(fs.readFile)
let fs_writeFile = util.promisify(fs.writeFile)

let get_css_less = pathname => /*await*/ build_cache.get(
  pathname,
  async result => {
    let render = await disk_build(
      pathname,
      async temp_pathname => {
        let render = await less.render(
          await fs_readFile(pathname, {encoding: 'utf-8'}),
          {
            compress: true,
            filename: pathname,
            pathnames: [path.posix.dirname(pathname)]
          }
        )
        await fs_writeFile(
          temp_pathname,
          render.css,
          {encoding: 'utf-8'}
        )
        return render.imports
      },
      true // diagnostics on
    )
    result.value = /*await*/ fs_readFile(render.pathname)
  }
}
```
We are relying on `fs_readFile()` or `less.render()` to throw exceptions if the
original stylesheet or any included stylesheet is not found or contains errors.

Also, note how much simplified the handling of asynchronicity is when using the
ES7 `async`/`await` syntax. We recommend to do this for all new code, and to do
it consistently, even if the use of `Promise` directly might give shorter code.

Code which is not already promisified, can be promisified as shown, or else we
can add specific conversions in places by code like: `await new Promise(...)`.
Note the comment `/* await */` where a `Promise` is passed through from a lower
level routine, an explicit `await` would be consistent here but less efficient.

## About diagnostics

If `diag` is passed as `true`, then a diagnostic will be printed indicating
whether an output file is being rebuilt, or an existing output is being reused.

## GIT repository

The development version can be cloned, downloaded, or browsed with `gitweb` at:
https://git.ndcode.org/public/disk_build.git

## License

All of our NPM packages are MIT licensed, please see LICENSE in the repository.

## Contributions

We would greatly welcome your feedback and contributions. The `disk_build` is
under active development (and is part of a larger project that is also under
development) and thus the API is considered tentative and subject to change. If
this is undesirable, you could possibly pin the version in your `package.json`.

Contact: Nick Downing <nick@ndcode.org>