Repository: YoWASP/yosys (language: python)

Issue:
Browser package (npm)
This was discussed on twitter a few times but I thought it would be helpful to track it here.

## Use Cases
* netlist visualization
  * type verilog in the browser and see the netlist update in real time
* resource usage visualization
  * I started working on this thing that would fetch fusesoc projects from github, synthesize them and display resource usage as a sunburst diagram
  * Running it client-side prevents users from using it as synthesis-as-a-service, we are using the users CPU cycles, not my CPU cycles
* Place and route visualization
  * I havent tried this but it would be interesting to play with 
* Simulation with cxxrtl
  * Useful for FPGA workshops where you dont want to deal with machine setup or user hardware

## Why
 * As a quick demo of Yosys. Some hardware engineers are not very computer savvy and explaining how to install python and running pip isn't as easy as sending them a link to a webpage. Seeing browser visualizations also makes for an impressive flashy demonstration. Also simply being able to run synthesis in browser demonstrates how light weight yoys is compared to vendor tools.
 * Introductory digital logic workshops are much easier when there is literally zero computer setup required
 * A lot of visualization tools are already in JS (netlistsvg, wavedrom, d3, etc)
 * People keep on doing one-off browser builds of Yosys for their specific use case. It would be nice if there was a good package that was good enough for everyone to use instead of rolling their own.

## What's the next best alternative?
 * Grab one of the builds of YosysJS floating around.
 * Jupyter notebooks are really great educational tool, but require learning the jupyter interface and at least a little python

## Challenges

Accessing files is annoying in the browser. In addition to the wasm file, you also need to deal with the users files, the tech library files. So, ideally those would be packaged in a way that is easy to use.


Discussion / hints:
This is possible and in scope. I have a prototype already and will look into publishing it as a complete work later.
> Place and route visualization

Note that Qt nextpnr builds are out of scope at the moment. That's an enormous and highly complex dependency that I am not ready to support.

> Simulation with cxxrtl

You'd have to also use clang.wasm targeting wasm. That's possible although the simulation speed might not be so great.
If there's anything I can do to help, I'm happy to contribute.
Not really, the limiting factor is my time.
@nturley https://github.com/YoWASP/yosys/pull/28

Fix patch:
diff --git a/.github/workflows/package.yml b/.github/workflows/package.yml
index e1fe527..b61730d 100644
--- a/.github/workflows/package.yml
+++ b/.github/workflows/package.yml
@@ -24,25 +24,37 @@ jobs:
         uses: actions/cache@v3
         with:
           path: ~/.cache/ccache
-          key: ${{ runner.os }}-
+          key: yosys-${{ hashFiles('yosys-src', 'SymbiYosys-src', 'build.sh') }}
+          restore-keys: |
+            yosys-${{ hashFiles('yosys-src', 'SymbiYosys-src', 'build.sh') }}
+            yosys-
       - name: Set up ccache
         run: |
           ccache --max-size=2G -z
       - name: Build WASM binaries
         run: |
           ./build.sh
-      - name: Build binary wheels
+      - name: Build Python artifact
         run: |
           ./package-pypi.sh
-      - name: Upload binary wheel artifact
+      - name: Upload Python artifact
         uses: actions/upload-artifact@v3
         with:
-          name: wheel
+          name: dist-pypi
           path: pypi/dist/
+      - name: Build JavaScript artifact
+        run: |
+          ./package-npmjs.sh
+          cp npmjs/test/test_api.mjs npmjs/dist/
+      - name: Upload JavaScript artifact
+        uses: actions/upload-artifact@v3
+        with:
+          name: dist-npmjs
+          path: npmjs/dist/
       - name: Print ccache statistics
         run: |
           ccache -s
-  test:
+  test-python:
     needs: build
     runs-on: ubuntu-latest
     strategy:
@@ -59,35 +71,57 @@ jobs:
       uses: actions/setup-python@v4
       with:
         python-version: '${{ matrix.python-version }}'
-    - name: Download binary wheel artifact
+    - name: Download Python artifact
       uses: actions/download-artifact@v3
       with:
-        name: wheel
+        name: dist-pypi
         path: dist/
-    - name: Test binary wheels
+    - name: Test Python artifact
       run: |
         pip install dist/*.whl
         yowasp-yosys --help
         yowasp-sby --help
         yowasp-yosys-smtbmc --help
         yowasp-yosys-witness --help
+  test-javascript:
+    needs: build
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        node-version:
+          - 'v18.x'
+          - 'v20.x'
+    steps:
+    - name: Set up Node.js
+      uses: actions/setup-node@v3
+      with:
+        node-version: '${{ matrix.node-version }}'
+    - name: Download JavaScript artifact
+      uses: actions/download-artifact@v3
+      with:
+        name: dist-npmjs
+        path: dist/
+    - name: Test JavaScript artifact
+      run: |
+        npm install ./dist/*.tgz
+        node dist/test_api.mjs
   check: # group all `test (*)` workflows into one for the required status check
-    needs: test
+    needs: [test-python, test-javascript]
     if: always() && !contains(needs.*.result, 'cancelled')
     runs-on: ubuntu-latest
     steps:
       - run: ${{ contains(needs.*.result, 'failure') && 'false' || 'true' }}
-  publish:
+  publish-python:
     needs: check
     runs-on: ubuntu-latest
     environment: publish
     permissions:
       id-token: write
     steps:
-    - name: Download binary wheel artifact
+    - name: Download Python artifact
       uses: actions/download-artifact@v3
       with:
-        name: wheel
+        name: dist-pypi
         path: dist/
     - name: Publish wheels to Test PyPI
       if: "github.event_name == 'push' && startsWith(github.event.ref, 'refs/heads/develop')"
@@ -97,6 +131,24 @@ jobs:
     - name: Publish wheels to PyPI
       if: "github.event_name == 'push' && startsWith(github.event.ref, 'refs/heads/release')"
       uses: pypa/gh-action-pypi-publish@release/v1
+  publish-javascript:
+    needs: check
+    runs-on: ubuntu-latest
+    if: "${{ startsWith(github.event.ref, 'refs/heads/develop') || startsWith(github.event.ref, 'refs/heads/release') }}"
+    steps:
+    - name: Set up Node.js
+      uses: actions/setup-node@v3
+      with:
+        registry-url: 'https://registry.npmjs.org'
+    - name: Download JavaScript artifact
+      uses: actions/download-artifact@v3
+      with:
+        name: dist-npmjs
+        path: dist/
+    - name: Publish package to NPM
+      run: npm publish --access public $(find dist -name *.tgz -printf 'file:%f ')
+      env:
+        NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
   release:
     needs: check
     runs-on: ubuntu-latest
diff --git a/README.md b/README.md
index e2f0c4e..587c085 100644
--- a/README.md
+++ b/README.md
@@ -3,39 +3,13 @@ YoWASP Yosys packages
 
 This package provides [Yosys][] binaries built for [WebAssembly][]. See the [overview of the YoWASP project][yowasp] for details.
 
+Two embeddings are provided: [Python](pypi/README.md) and [JavaScript](npmjs/README.md).
+
 [yosys]: https://github.com/YosysHQ/yosys/
 [webassembly]: https://webassembly.org/
 [yowasp]: https://yowasp.github.io/
 
 
-Versioning
-----------
-
-The version of this package is derived from the upstream Yosys package version in the ``X.Y[.Z][+N]`` format, and is comprised of five or six parts in a ``X.Y.Z.N.postM[.dev]`` format:
-
-1. ``X``: Yosys major version
-2. ``Y``: Yosys minor version
-3. ``Z``: Yosys patch version; only present for some Yosys releases, zero if not present
-4. ``N``: zero for packages built from Yosys releases, ``N`` for packages built from unreleased Yosys snapshots; matches the ``N`` in the ``X.Y+N`` upstream version
-5. ``postM``: package build version; disambiguates different builds produced from the same Yosys source tree
-6. ``dev``: present only for packages built from unreleased Yosys snapshots; marks these packages as pre-releases
-
-With this scheme, there is a direct correspondence between upstream versions and [PEP 440][pep440] Python package versions. Packages built from unreleased snapshots are ignored by pip by default, but can be still installed explicitly. (These packages are uploaded daily to [TestPyPI][], but only occasionally to [PyPI][].)
-
-A different versioning scheme was used earlier, where the package build version was denoted by a ``.devM`` suffix. This scheme did not work well with [PEP 440 version specifiers][pep440-vs] and was retired.
-
-[testpypi]: https://test.pypi.org/
-[pypi]: https://pypi.org/
-[pep440]: https://peps.python.org/pep-0440/
-[pep440-vs]: https://peps.python.org/pep-0440/#version-specifiers
-
-
-Configuration
--------------
-
-See the documentation for [yowasp-runtime](https://github.com/YoWASP/runtime#configuration).
-
-
 License
 -------
 
diff --git a/npmjs/.gitignore b/npmjs/.gitignore
new file mode 100644
index 0000000..6b79fc8
--- /dev/null
+++ b/npmjs/.gitignore
@@ -0,0 +1,7 @@
+/package-lock.json
+/package.json
+/node_modules
+/dist
+
+/*.wasm
+/gen
diff --git a/npmjs/README.md b/npmjs/README.md
new file mode 100644
index 0000000..6266736
--- /dev/null
+++ b/npmjs/README.md
@@ -0,0 +1,47 @@
+YoWASP Yosys package
+====================
+
+This package provides [Yosys][] binaries built for [WebAssembly][]. See the [overview of the YoWASP project][yowasp] for details.
+
+At the moment, this package only provides an API allowing to run Yosys in a virtual filesystem; no binaries are provided.
+
+This package does not provide access to `sby`, `yowasp-yosys-smtbmc`, and `yowasp-yosys-witness`; only the main `yosys` application is available.
+
+[yosys]: https://github.com/YosysHQ/yosys/
+[webassembly]: https://webassembly.org/
+[yowasp]: https://yowasp.github.io/
+
+
+API reference
+-------------
+
+This package provides one function:
+
+- `runYosys` (alias `cmd:yosys`)
+
+For more detail, see the documentation for [the JavaScript YoWASP runtime](https://github.com/YoWASP/runtime-js#api-reference).
+
+
+Versioning
+----------
+
+The version of this package is derived from the upstream Yosys package version in the `X.Y[.Z][+N]` format, and can be in one of the two formats, `X.Y.M` (for builds from release branches) or `X.(Y+1).N-dev.M` (for development builds), where the symbols are:
+
+1. `X`: Yosys major version
+2. `Y`: Yosys minor version
+3. `Z`: Yosys patch version; ignored if present
+4. `N`: matches the `N` in the `X.Y+N` upstream version, if present
+5. `M`: package build version; disambiguates different builds produced from the same Yosys source tree
+6. `-dev`: present only for packages built from unreleased Yosys snapshots; marks these packages as pre-releases
+
+With this scheme, there is a direct correspondence between upstream versions and [SemVer][semver] NPM package versions.
+
+Note that for development builds, the minor version is incremented as required by the SemVer comparison order. This means that an NPM package built from the upstream version `0.45+12` and the 120th commit in this repository will have the version `0.46.12-dev.120`. Because this is a pre-release package, it will not be matched by version specifiers such as `^0.46` or `>=0.46`.
+
+[semver]: https://semver.org/
+
+
+License
+-------
+
+This package is covered by the [ISC license](LICENSE.txt), which is the same as the [Yosys license](https://github.com/YosysHQ/yosys/blob/master/COPYING).
diff --git a/npmjs/gen/.gitkeep b/npmjs/gen/.gitkeep
new file mode 100644
index 0000000..e69de29
diff --git a/npmjs/lib/api.js b/npmjs/lib/api.js
new file mode 100644
index 0000000..d367e77
--- /dev/null
+++ b/npmjs/lib/api.js
@@ -0,0 +1,17 @@
+import { Application } from '@yowasp/runtime';
+import { instantiate } from '../gen/yosys.js';
+
+export { Exit } from '@yowasp/runtime';
+
+// Paths are relative to the bundle, which lives in gen/ together with the other build products.
+const yosys = new Application(import.meta.url, {
+    'share': './yosys-share.json',
+}, {
+    'yosys.core.wasm': './yosys.core.wasm',
+    'yosys.core2.wasm': './yosys.core2.wasm',
+    'yosys.core3.wasm': './yosys.core3.wasm',
+    'yosys.core4.wasm': './yosys.core4.wasm',
+}, instantiate, 'yowasp-yosys');
+
+export const runYosys = yosys.run.bind(yosys);
+export { runYosys as 'cmd:yosys' };
diff --git a/npmjs/package-in.json b/npmjs/package-in.json
new file mode 100644
index 0000000..0b80df5
--- /dev/null
+++ b/npmjs/package-in.json
@@ -0,0 +1,36 @@
+{
+    "name": "@yowasp/yosys",
+    "version": "__VERSION__",
+    "description": "Yosys Open SYnthesis Suite",
+    "author": "Catherine <whitequark@whitequark.org>",
+    "license": "ISC",
+    "homepage": "https://yowasp.org/",
+    "repository": {
+        "type": "git",
+        "url": "git+https://github.com/YoWASP/yosys.git"
+    },
+    "bugs": {
+        "url": "https://github.com/YoWASP/yosys/issues"
+    },
+    "type": "module",
+    "files": [
+        "gen/*.json",
+        "gen/*.wasm",
+        "gen/bundle-*.js"
+    ],
+    "exports": {
+        "node": "./gen/bundle-node.js",
+        "browser": "./gen/bundle-browser.js"
+    },
+    "devDependencies": {
+        "@bytecodealliance/jco": "^0.14.1",
+        "@yowasp/runtime": "3.1.14",
+        "esbuild": "^0.19.8"
+    },
+    "scripts": {
+        "pack": "yowasp-pack-resources ../yosys-build/share gen/yosys-share.json",
+        "transpile": "jco new ../yosys-build/yosys.wasm --wasi-command --output yosys.wasm && jco transpile yosys.wasm --instantiation async --no-typescript --no-namespaced-exports --map 'wasi:io/*=runtime#io' --map 'wasi:cli/*=runtime#cli' --map 'wasi:clocks/*=runtime#*' --map 'wasi:filesystem/*=runtime#fs' --out-dir gen/",
+        "build:node": "esbuild --bundle lib/api.js --outfile=gen/bundle-node.js --format=esm --platform=node",
+        "build:browser": "esbuild --bundle lib/api.js --outfile=gen/bundle-browser.js --format=esm --platform=browser"
+    }
+}
diff --git a/npmjs/prepare.py b/npmjs/prepare.py
new file mode 100644
index 0000000..145b76e
--- /dev/null
+++ b/npmjs/prepare.py
@@ -0,0 +1,35 @@
+import os
+import re
+import json
+import subprocess
+
+
+yosys_version_raw = subprocess.check_output([
+    "make", "-s", "-C", "../yosys-src", "echo-yosys-ver"
+], encoding="utf-8").strip()
+
+git_rev_list_raw = subprocess.check_output([
+    "git", "rev-list", "HEAD"
+], encoding="utf-8").split()
+
+# Yosys can't figure out if it should have a patch version or not.
+# Match one, and add one below in our version just in case.
+yosys_version = re.match(r"^(\d+)\.(\d+)(?:\.(\d+))?(?:\+(\d+))?$", yosys_version_raw)
+yosys_major  = int(yosys_version[1])
+yosys_minor  = int(yosys_version[2])
+_yosys_patch = int(yosys_version[3] or "0")
+yosys_node   = int(yosys_version[4] or "0")
+
+distance = len(git_rev_list_raw) - 1
+
+if os.environ.get("RELEASE_BRANCH", "false") in ("true", "1", "yes"):
+    version = f"{yosys_major}.{yosys_minor}.{distance}"
+else:
+    version = f"{yosys_major}.{yosys_minor + 1}.{yosys_node}-dev.{distance}"
+print(f"version {version}")
+
+with open("package-in.json", "rt") as f:
+    package_json = json.load(f)
+package_json["version"] = version
+with open("package.json", "wt") as f:
+    json.dump(package_json, f)
diff --git a/package-npmjs.sh b/package-npmjs.sh
new file mode 100755
index 0000000..2153c40
--- /dev/null
+++ b/package-npmjs.sh
@@ -0,0 +1,15 @@
+#!/bin/sh -ex
+
+PYTHON=${PYTHON:-python}
+
+cd $(dirname $0)/npmjs
+
+${PYTHON} prepare.py
+npm install
+npm run pack
+npm run transpile
+npm run build:node
+npm run build:browser
+
+mkdir -p dist
+npm pack --pack-destination dist
\ No newline at end of file
diff --git a/pypi/README.md b/pypi/README.md
deleted file mode 120000
index 32d46ee..0000000
--- a/pypi/README.md
+++ /dev/null
@@ -1,1 +0,0 @@
-../README.md
\ No newline at end of file
diff --git a/pypi/README.md b/pypi/README.md
new file mode 100644
index 0000000..7d4d4bc
--- /dev/null
+++ b/pypi/README.md
@@ -0,0 +1,42 @@
+YoWASP Yosys package
+====================
+
+This package provides [Yosys][] binaries built for [WebAssembly][]. See the [overview of the YoWASP project][yowasp] for details.
+
+[yosys]: https://github.com/YosysHQ/yosys/
+[webassembly]: https://webassembly.org/
+[yowasp]: https://yowasp.github.io/
+
+
+Versioning
+----------
+
+The version of this package is derived from the upstream Yosys package version in the `X.Y[.Z][+N]` format, and is comprised of five or six parts in a `X.Y.Z.N.postM[.dev]` format, where the symbols are:
+
+1. `X`: Yosys major version
+2. `Y`: Yosys minor version
+3. `Z`: Yosys patch version; only present for some Yosys releases, zero if not present
+4. `N`: zero for packages built from Yosys releases, `N` for packages built from unreleased Yosys snapshots; matches the `N` in the `X.Y+N` upstream version
+5. `postM`: package build version; disambiguates different builds produced from the same Yosys source tree
+6. `dev`: present only for packages built from unreleased Yosys snapshots; marks these packages as pre-releases
+
+With this scheme, there is a direct correspondence between upstream versions and [PEP 440][pep440] Python package versions. Packages built from unreleased snapshots are ignored by pip by default, but can be still installed explicitly. (These packages are uploaded daily to [TestPyPI][], but only occasionally to [PyPI][].)
+
+A different versioning scheme was used earlier, where the package build version was denoted by a `.devM` suffix. This scheme did not work well with [PEP 440 version specifiers][pep440-vs] and was retired.
+
+[testpypi]: https://test.pypi.org/
+[pypi]: https://pypi.org/
+[pep440]: https://peps.python.org/pep-0440/
+[pep440-vs]: https://peps.python.org/pep-0440/#version-specifiers
+
+
+Configuration
+-------------
+
+See the documentation for [the Python YoWASP runtime](https://github.com/YoWASP/runtime-py#configuration).
+
+
+License
+-------
+
+This package is covered by the [ISC license](LICENSE.txt), which is the same as the [Yosys license](https://github.com/YosysHQ/yosys/blob/master/COPYING).


Test patch:
diff --git a/npmjs/test/test_api.mjs b/npmjs/test/test_api.mjs
new file mode 100644
index 0000000..8890636
--- /dev/null
+++ b/npmjs/test/test_api.mjs
@@ -0,0 +1,78 @@
+import { runYosys } from '@yowasp/yosys';
+
+runYosys(["--version"]);
+
+const files = await runYosys(
+  ["inv.v", "-p", "hierarchy -auto-top", "-o", "top.json"],
+  {"inv.v": "module inv(input a, output o); assign o = ~a; endmodule"});
+const actual = JSON.parse(files["top.json"]);
+const expected = {
+  "creator": actual["creator"],
+  "modules": {
+    "inv": {
+      "attributes": {
+        "hdlname": "\\inv",
+        "top": "00000000000000000000000000000001",
+        "src": "inv.v:1.1-1.56"
+      },
+      "ports": {
+        "a": {
+          "direction": "input",
+          "bits": [ 2 ]
+        },
+        "o": {
+          "direction": "output",
+          "bits": [ 3 ]
+        }
+      },
+      "cells": {
+        "$not$inv.v:1$1": {
+          "hide_name": 1,
+          "type": "$not",
+          "parameters": {
+            "A_SIGNED": "00000000000000000000000000000000",
+            "A_WIDTH": "00000000000000000000000000000001",
+            "Y_WIDTH": "00000000000000000000000000000001"
+          },
+          "attributes": {
+            "src": "inv.v:1.43-1.45"
+          },
+          "port_directions": {
+            "A": "input",
+            "Y": "output"
+          },
+          "connections": {
+            "A": [ 2 ],
+            "Y": [ 3 ]
+          }
+        }
+      },
+      "netnames": {
+        "$not$inv.v:1$1_Y": {
+          "hide_name": 1,
+          "bits": [ 3 ],
+          "attributes": {
+            "src": "inv.v:1.43-1.45"
+          }
+        },
+        "a": {
+          "hide_name": 0,
+          "bits": [ 2 ],
+          "attributes": {
+            "src": "inv.v:1.18-1.19"
+          }
+        },
+        "o": {
+          "hide_name": 0,
+          "bits": [ 3 ],
+          "attributes": {
+            "src": "inv.v:1.28-1.29"
+          }
+        }
+      }
+    }
+  }
+};
+if (JSON.stringify(actual) !== JSON.stringify(expected)) {
+  throw 'test failed';
+}
\ No newline at end of file