npm package: @normed/bundle
Description: Output directory structure is *the* first class citizen in this used-in-production PoC bundler.
Author: Normal Gaussian
Latest version: 4.8.8

# Premise

Output directory structure is *the* first class citizen in this used-in-production PoC bundler.

By default, entrypoints are denoted by the `static` modifier, a browser suitable `tsconfig` is applied to typescript files whose entrypoint has the `web` prefix, and a node `tsconfig` is used for typescript files whose entrypoint has the `node` prefix. The `lib` prefix does not bundle files.

There is the possibility to configure specific endpoints and configs using a settings json in `package.json` or `.bundle.config.json`, but the user is encouraged to configure modifiers instead and allow entrpoints to be auto-discovered.

For example:

```
in:                                 out:
  src/                                bundles/
  ├─ api.ts                           |
  ├─ cli.node.static.ts               ├─ cli.js
  ├─ logger.ts                        |
  ├─ index.static.ts                  ├─ index.js
  ├─ public/                          ├─ public/
     ├─ base.style.less                  |
     ├─ favicon.static.ico               ├─ favicon.ico
     ├─ index.static.less                ├─ index.css
     ├─ simple.template.pug              |
     ├─ index.static.pug                 ├─ index.html
     ├─ index.web.static.ts              ├─ index.js
```

The non-static files are removed, and the entrypoints are auto-discovered. The `cli.js` and `public/index.js` have been bundled with all of their dependencies, but `index.js` has not.

# Things to note

The following global javascript variables are available in most environments:
* `__filename` and `__dirname` - refers to the executing bundle filename/dirname
The following additional variables provided by `@normed/bundle`, and do not exist by default if not built by `@normed/bundle`:
* `__source_filename` and `__source_dirname` - refer to the original source files absolute filename/dirname
* `__source_relative_filename` and `__source_relative_dirname` - refer to the original source files filename/direnation relative to the source directory

The current version of `@normed/bundle` is uses es-build for speed, and then slows it down with all the additional behaviours written in TypeScript.

# Configuration

Specify your configuration in your `package.json` by adding the `"bundle"` entry.

```ts
  "bundle"?: {
    "in"?: string,
    "out"?: string,
    // A list of entrypoints, with or without a specific config.
    "entrypoints"?: ({ "path": string, "config"?: EntryConfig} | { "path": string, "modifiers"?: string[] } | string)[],
    // Define what each modifier does
    "modifiers"?: {
      [modifier: string]: EntryConfig
    },
    // Full config to override the default config
    "baseConfig"?: EntryConfig
  }
```

```ts
type EntryConfig = {
  "entrypoint"?: boolean, 

  // When true or a string outputs bundle analysis files. When a string, uses that string to prefix analysis files
  "analyse"?: boolean | string,
  
  // A shorthand to extend another known modifier
  "extends"?: string,

  // Configuration specific to typescript
  "typescript"?: {
    // If a string, the location of a tsconfig file to use.
    // If a tsConfig object, a tsconfig object to use // TODO: determine how extension should work in this case
    "tsconfig"?: string | TsConfig,

    // Whether to generate declarations files
    // default false
    "declarations"?: boolean,
  }

  // Configuration specific to javascript output (e.g. includes typescript input)
  "javascript"?: {
    "esbuild"?: {
      // An ESBuildConfig object to use
      "config"?: string | ESBuildConfig,
    }
  }

  // Configuration specific to CSS processing
  "css"?: {
    // URL patterns to mark as external (not resolved at build time)
    // Supports glob-style wildcards, e.g. ["/fonts/*", "/images/*"]
    "externalUrls"?: string[]
  }
}
```

Or if calling via the command line:
```sh
bundle --tsc-external

bundle --tsc-declarations

bundle --indir <directory>

bundle --outdir <directory>

bundle --platform <browser | web | node | server | library>

bundle --platform server --platform-modifier node
```
### bundling

By default this utility exports the `bundle` command; this command defaults to building mode, but it can be explicitly set if desired.

```sh
# These two commands are equivalent
yarn bundle
yarn bundle --build
```

### cleaning

If you want bundle to delete the contents of the output directory, you can tell it to clean it:

```sh
yarn bundle --clean
```

### watch mode (not yet implemented)

Watch mode is not yet implemented.

### Logging

Increase logging with the `--verbose` and `--debug` flags. Decrease logging with the `--quiet` flag.

```sh
yarn bundle --verbose
yarn bundle -v

yarn bundle --debug

yarn bundle --quiet
yarn bundle -q
```

Alternatively, explicitly set the log level with the `LOG_LEVEL` environment variable. Suitable values are `verbose`, `debug`, `info`, `warn`, `error`.

### entrypoints

By default files ending in `.static.<ext>`, `.node.<ext>`, and `.web.<ext>` are automatically picked up as entrypoints.

Alternate entrypoints can be set using the `--platform-modifier` CLI option.

Any files you don't want to name with such a suffix can be added manually. To add the files `anchovies.ts` and `pics/cartoons.png` manually:

*via `package.json`:*
```json
{
  "bundle": {
    "entrypoints": [ "anchovies.ts", "pics/cartoons.png" ]
  }
}
```

*via cli:*
```sh
yarn bundle --entrypoint anchovies.ts --entrypoint pics/cartoons.png
```

### asset modifier

Use the `.asset` modifier to copy files verbatim without any processing. This is useful for pre-built CSS or JavaScript files that should not be processed by esbuild:

```
src/
├─ vendor/
│  ├─ legacy.asset.js      → bundles/vendor/legacy.js  (copied verbatim)
│  ├─ styles.asset.css     → bundles/vendor/styles.css (copied verbatim)
```

This is particularly helpful for:
- Third-party CSS/JS libraries with complex `url()` references
- Pre-minified vendor files
- Files with runtime paths that shouldn't be resolved at build time

### default file loaders

Common asset types are automatically configured to use esbuild's `file` loader:
- Images: `.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`, `.svg`, `.ico`
- Fonts: `.ttf`, `.otf`, `.woff`, `.woff2`, `.eot`
- Media: `.mp4`, `.webm`, `.mp3`, `.wav`

This means these files referenced from your CSS/JS will be copied to the output directory and their paths will be rewritten appropriately.

### asset paths

Control how static assets (images, fonts, media) are named and referenced in your output.

```json
{
  "bundle": {
    "assetNames": "assets/[name]-[hash]",
    "publicPath": "https://cdn.example.com"
  }
}
```

- **`assetNames`** — Template for output filenames. Supports `[name]`, `[hash]`, `[ext]`, `[dir]`. Default: `[name]-[hash]`.
- **`publicPath`** — URL prefix prepended to asset references in HTML and CSS. Useful for CDNs or when assets are served from a specific path.

**Note:** `publicPath` only affects static assets (images, fonts, media). Links between HTML pages, CSS stylesheets, and JavaScript files discovered in pug templates always use relative paths.

The default `assetNames` is `[name]-[hash]` for static assets in pug templates. For CSS/JS files discovered in pug templates, the default is `[dir]/[name]-[hash]` to preserve directory structure.

#### Serving from a subdirectory (`webRoot`)

If your output has a subdirectory that serves as the web root (common when bundling both server and client code), set `webRoot` so asset URLs are computed relative to it:

```json
{
  "bundle": {
    "webRoot": "public",
    "assetNames": "public/assets/[name]-[hash]"
  }
}
```

```
src/                              bundles/
├─ public/                        ├─ public/
│  ├─ index.static.pug    →      │  ├─ index.html
│  ├─ logo.png             →     │  ├─ assets/logo-A1B2C3D4.png
├─ worker.node.ts          →     ├─ worker.js
```

With `webRoot: "public"`, `index.html` references the logo as `/assets/logo-A1B2C3D4.png` — correct for a server serving `bundles/public/` at `/`.

Without `webRoot`, the reference would incorrectly be `/assets/public/assets/logo-A1B2C3D4.png`.

To host under a URL prefix, combine with `publicPath`:

```json
{
  "bundle": {
    "webRoot": "public",
    "publicPath": "/myapp",
    "assetNames": "public/assets/[name]-[hash]"
  }
}
```

This produces URLs like `/myapp/assets/logo-A1B2C3D4.png`.

> **Note:** These options can also be set under `baseConfig.javascript.esbuild` for backwards compatibility, but the top-level location is preferred.

### external CSS URLs

For CSS files that reference runtime paths (like `/fonts/*` or `/images/*`) that should not be resolved at build time, use the `css.externalUrls` configuration:

```json
{
  "bundle": {
    "baseConfig": {
      "css": {
        "externalUrls": ["/fonts/*", "/images/*", "https://*"]
      }
    }
  }
}
```

URL patterns support glob-style wildcards (`*`). URLs matching these patterns will be marked as external and left unchanged in the output CSS.

### pug-to-pug linking

When a pug file contains links to other pug files, those linked files are automatically discovered and built as entrypoints. The href attributes are rewritten to point to the compiled HTML output.

```pug
//- index.static.pug
html
  body
    nav
      a(href="about.pug") About Us
      a(href="pages/contact.pug") Contact
```

This produces:
```
src/                          bundles/
├─ index.static.pug    →     ├─ index.html (href rewritten to "about.html")
├─ about.pug           →     ├─ about.html (auto-discovered)
├─ pages/                    ├─ pages/
   └─ contact.pug      →        └─ contact.html (auto-discovered)
```

Features:
- **Recursive**: If `about.pug` links to `team.pug`, that gets built too
- **Circular reference safe**: Files are only processed once
- **Asset handling**: Images and other assets in discovered pug files are processed normally
- **Warning on missing**: A warning is logged if a referenced pug file doesn't exist

This works for all HTML attributes that can contain paths (href, src, data, poster, etc.).

### Node-resolved references (`build:` / `copy:`)

Pug templates support two prefixes for referencing files via Node's module resolution (`require.resolve`):

| Prefix | Action |
|--------|--------|
| `build:` | Resolve via Node, then compile through the appropriate pipeline (Less→CSS, Pug→HTML, TS→JS) |
| `copy:` | Resolve via Node, then copy to output with content-hashing |

```pug
//- Compile a Less file from a shared location
link(rel="stylesheet" href="build:#shared/styles.less")

//- Copy a vendor CSS file (url() references are rewritten)
link(rel="stylesheet" href="copy:normalize.css/normalize.css")

//- Copy a vendor JS file as-is
script(src="copy:bootstrap/dist/js/bootstrap.min.js")
```

Both prefixes support:
- **Bare specifiers**: `bootstrap/dist/css/bootstrap.css` — resolves from `node_modules`
- **`#` subpath imports**: `#shared/styles.less` — resolves via the `imports` field in `package.json`

#### Configuring `#` subpath imports

Add an `imports` field to your `package.json`:

```json
{
  "imports": {
    "#shared/*": "./src/shared/*",
    "#assets/*": "./src/assets/*"
  }
}
```

This is a standard Node.js feature (v12.19+) that also works natively in esbuild and TypeScript (4.5+). One config, consistent everywhere.

#### `build:` supported file types

| Extension | Pipeline |
|-----------|----------|
| `.less` | Less → CSS (via esbuild CSS pipeline) |
| `.pug` | Pug → HTML |
| `.ts`, `.tsx`, `.js`, `.jsx`, `.mjs`, `.cjs` | TypeScript/JavaScript bundling |
| `.css` | CSS (via esbuild CSS pipeline, handles `url()` rewriting) |

Using `build:` on an unsupported extension emits a warning suggesting `copy:` instead.

#### Deprecation of `pkg:`

The `pkg:` prefix is deprecated in favour of `copy:`. Both work identically — `pkg:` emits a deprecation warning during build. Replace `pkg:normalize.css/normalize.css` with `copy:normalize.css/normalize.css`.

### analyse

To get a report on bundle content and size enable analysis:

*`package.json`*:
```json
{
  "bundle": {
    "analyse": true
  }
}
```

*via cli:*
```sh
bundle --analyse
```

# Development

Feel free to contribute to the codebase, issues and code welcome.

## build pipeline

`@normed/bundle` is bootstrapped by being built from typescript to javascript first (using `tsc`, output to `dist`), then using the javascript output to build the original source (output to `bundles-a`), then using that output to build the original source again (output to `bundles`), and finally the last two outputs are compared to ensure they are the same. This acts as a nice little test to check some of the functionality of `@normed/bundle`.

## custom builders

`@normed/bundle` uses the concept of "builders" to map file extensions to output files. There are default builders for `.pug`, `.less`, `.js`, and `.ts`; with all other entrpoints being copied. These are specified in `src/builders.ts`.

To use custom builders you will need to invoke bundle via code rather than the CLI, and specify your additional builders.

```ts
import { bundle } from '@normed/bundle';
import { additionalBuilders } from './myAdditionalBuilders';

bundle({
  builders: additionalBuilders
})
```

The default builders are automatically added to your builders. Builders match files based on their poisition in the builder list.

To write a builder, consult the existing builders in `builders.ts` and the `Builder` type in `types.ts`. Also consider:

### basic fields

* Its `name` should be clear and, ideally, short. This will appear in logs
* The `color` is constructed using the module 'chalk'. `chalk.hex(...)` works. This (only) appears in logs, so you should probably avoid red/yellow/green.
* `inExt` is a list of file extensions to match against, either as strings or regular expressions. If you use regular expressions they have to be fully anchored (start with `^` and end with `$`).
* `outExt` is the extension the rewritten filename will have in the `outfile_absolute` and `outfile_relative` variables to the `build` option.

### build field

The most important field is of course `build`.

It does the real work. It has the signature:

```ts
(entrypoints: Entrypoint[], extra: BuildConfig["extra"]): Promise<void>
```

All of the entrypoints that use your builder will be provided at once - this is because some builder methods are much faster and simpler this way! If that isn't the case for yours, just iterate over them.

The extra field contains additional configuration from the `package.json` build config. This hasn't been set up yet to correctly handle plugins, so you probably don't want to use this. If you do you will likely have to do a force cast of the types.

Each entrypoint has a few fields you will care about:

* `platform` - currently one of `web`, `server`, or `library`. This indicates the expected runtime environment for the file.

* `{infile|outfile}_{relative|absolute}` - to simplify path mangling, the input and output files have been worked out already. Please use relative paths in logs, and anywhere where they may affect reproducible builds. Ensure your result is written to the outfile path!

* `inbase` - this is the absolute path of the root directory for building. Often it is not the root directory of a repo, but a subfolder like `src`. This is often useful to set as the `cwd` for any execution. 

Currently we expect each builder to write its results to disk. There may be a point in the future where we have the builder provide the results as a return structure, however it is more likely that we will use multiple instances of overlayFS to handle this type of behaviour.

### testing

There is a verify builder function exported:

```ts
import { verifyBuilder } from '@normed/bundle';
```

This does not fully test your builder, but it does run some tests we have found helpful.

# Common Pitfalls

### Custom entrypoint modifiers not discovered

**Problem**: You want to use a custom file suffix like `.worker.ts` but files aren't being discovered.

**Cause**: Using `platformModifiers` in package.json. This is a CLI-only option, not a package.json config.

**Solution**: Use `modifiers` with `entrypoint: true`:

```json
{
  "bundle": {
    "modifiers": {
      "worker": {
        "entrypoint": true,
        "extends": "server"
      }
    }
  }
}
```

Now `*.worker.ts` files will be discovered and built with server platform settings.

### No entrypoints found

**Problem**: Build completes but reports "No entrypoints".

**Cause**: Files must match a recognized pattern (`.static.*`, `.node.*`, `.web.*`, or custom modifiers with `entrypoint: true`).

**Solution**: Either:
1. Rename your file to use a recognized pattern (e.g., `index.static.ts`)
2. Add a custom modifier in package.json (see above)
3. Manually specify entrypoints: `"entrypoints": ["myfile.ts"]`

Use `LOG_LEVEL=verbose yarn bundle` to see which patterns are being searched.

# Known issues

### Declaration generation may resolve wrong dependency versions

When `declarations: true` is set, the bundler uses TypeScript's compiler API (`ts.createProgram`) to generate `.d.ts` files. In Yarn PnP environments, this can resolve `@types` packages to the wrong version — e.g. picking up a transitive `@types/node@20` instead of your workspace's direct `@types/node@25`.

This happens because `ts.createCompilerHost` uses filesystem-walking resolution, which goes through PnP's `fs` patch (layer 2) rather than PnP's scope-aware `require` resolution (layer 1). The `fs` patch makes all zip cache contents readable but doesn't enforce dependency scoping, so transitive dependencies can leak into type resolution. The Yarn TypeScript SDK solves this for `tsc` and `tsserver` by wrapping them to call `.pnp.cjs.setup()`, but the programmatic `ts.createProgram` API doesn't benefit from this.

If you encounter spurious type errors during declaration generation that don't appear in `tsc` or your IDE, this is likely the cause. A workaround is to fix the offending type at the call site.

# Potential errors

es-build, which is essentially the basis for this whole thing, breaks reproducible builds as it includes the full path to a file as both a comment and a map key. To get around this we manually replace certain paths on build. There is a good chance this will not work for globally installed modules and will break certain hard coded strings.