Repository: cocotb/cocotb

Issue:
Distribute wheels on PyPI
We can now distribute cocotb as a wheel, except for FLI support (well maybe we can, need to talk to Mentor about that). I suggest we provide the following matrix of wheels at first:

* OS
  * Linux (`manylinux1` supports C++11)
  * Mac OS X
  * Windows-MinGW
* Arch
  * x86_64
* Python version
   * CPython 3.X `cp3`

These packages should be built and uploaded in CI on release candidate and final releases. May need to investigate distributing with Github to work around the issue of Pypi not supporting nightlies.


Discussion:
I'm looking into that now as part of the release upload CI. Seems to be not super straight-forward, though.
Yeah, it's not. For manylinux1 distributions (which support C++11), you need to build using RHEL6 image, there are official docker images that are typically used. 'auditwheel' can check manylinux1 support. Windows and Mac wheels are not standardized. Extension modules are built to only work with particular version of python, so we need potentially multiple builds per arch. And of course there is the issue of FLI support. 

There is [cibuildwheel](https://github.com/joerick/cibuildwheel) and [python-wheels-manylinux-build](https://github.com/RalfG/python-wheels-manylinux-build) which should help.

I was thinking it would be useful to provide wheels for platforms that are more difficult to support (Windows) and/or very common platforms like standard CPython 3.5-3.9 on Linux. And for now leave the other platforms for source distribution only. And I think we can safely support only x86_64 only for now.
Yeah, cibuildwheel should do most of the trick. However, we really need to sort out the FLI stuff. If we provide wheels and users want to use a Mentor simulator, what do they do? If we don't provide a "recompile on first use" option, users would need to re-install without wheels? That doesn't feel very easy to understand from a user's perspective, so I'd argue they'd be better off with no wheels at all.
I know the FLI and Questa headers are not licensed in a way we can distribute, but I'm not sure about how they feel about distributing binaries built from that source. It might be possible to build FLI in private CI.

>  If we don't provide a "recompile on first use" option, users would need to re-install without wheels?

Users currently have to do that if they install the first time without Questa in the path. You have to force reinstall with the wheel cache off or it will simply reinstall the cached wheel without FLI support again. So we are already suffering from that problem in a way. Our current solution is broken IMO.

Personally, I think the best way forward is to separate out the FLI into an extension (`pip install cocotb[fli]`) and have the extension __require__ the FLI headers to be picked up. That way no faulty wheel without FLI support can be made, and a built wheel can be reused without FLI being on the path during installation.
Making it a separate extension might do the trick if we cannot reach anybody on the Mentor side. I'll try to do that, but for the 1.5 release all of that feels too risky, so let's continue distributing without wheels for now, and pick this discussion up after we have clarity what we can and cannot do with mti.h
> And I think we can safely support only x86_64 only for now.

GHDL and VUnit work on ARM, either RPi, Zynq (Pynq) or smartphones/tables (Android/Termux). Although not first priority, I think it would be very desirable to have cocotb available on ARM (at least aarch64) targets.
Making wheels and supporting the environment in the first place are different efforts, you might be getting ahead of yourself 😄.

Resolving patch:
diff --git a/noxfile.py b/noxfile.py
--- a/noxfile.py
+++ b/noxfile.py
@@ -2,6 +2,7 @@
 # Licensed under the Revised BSD License, see LICENSE for details.
 # SPDX-License-Identifier: BSD-3-Clause
 import glob
+import sys
 from contextlib import suppress
 from pathlib import Path
 from typing import Dict, List, Optional, Tuple
@@ -266,6 +267,134 @@ def dev_coverage_report(session: nox.Session) -> None:
     session.run("gcovr", "--print-summary", "--txt")
 
 
+#
+# Release pipeline.
+#
+# - Build wheels (release builds).
+# - Install cocotb from wheel.
+# - Run tests against cocotb installed from the wheel.
+#
+# The release pipeline does not collect coverage, and does not run doctests.
+#
+
+# Directory containing the distribution artifacts (sdist and bdist).
+dist_dir = "dist"
+
+
+@nox.session
+def release_build(session: nox.Session) -> None:
+    """Build a release (sdist and bdist)."""
+    session.notify("release_build_bdist")
+    session.notify("release_build_sdist")
+
+
+@nox.session
+def release_build_bdist(session: nox.Session) -> None:
+    """Build a binary distribution (wheels) on the current operating system."""
+
+    # Pin a version to ensure reproducible builds.
+    session.install("cibuildwheel==2.5.0")
+
+    # cibuildwheel only auto-detects the platform if it runs on a CI server.
+    # Do the auto-detect manually to enable local runs.
+    if sys.platform.startswith("linux"):
+        platform = "linux"
+    elif sys.platform == "darwin":
+        platform = "macos"
+    elif sys.platform == "win32":
+        platform = "windows"
+    else:
+        session.error(f"Unknown platform: {sys.platform!r}")
+
+    session.log("Building binary distribution (wheels)")
+    session.run(
+        "cibuildwheel",
+        "--platform",
+        platform,
+        "--output-dir",
+        dist_dir,
+    )
+
+    session.log(
+        f"Binary distribution in release mode for {platform!r} built into {dist_dir!r}"
+    )
+
+
+@nox.session
+def release_build_sdist(session: nox.Session) -> None:
+    """Build the source distribution."""
+
+    session.install("build")
+
+    session.log("Building source distribution (sdist)")
+    session.run("python", "-m", "build", "--sdist", "--outdir", dist_dir, ".")
+
+    session.log(f"Source distribution in release mode built into {dist_dir!r}")
+
+
+def release_install(session: nox.Session) -> None:
+    """Helper: Install cocotb from wheels and also install test dependencies."""
+
+    session.log(f"Installing cocotb from wheels in {dist_dir!r}")
+    session.install(
+        "--only-binary",
+        "cocotb",
+        "--no-index",
+        "--find-links",
+        dist_dir,
+        "cocotb",
+    )
+
+    session.log("Running cocotb-config as basic installation smoke test")
+    session.run("cocotb-config", "--version")
+
+    session.log("Installing test dependencies")
+    session.install(*test_deps)
+
+
+@nox.session
+@nox.parametrize("sim,toplevel_lang,gpi_interface", simulator_support_matrix())
+def release_test_sim(
+    session: nox.Session, sim: str, toplevel_lang: str, gpi_interface: str
+) -> None:
+    """Test a release version of cocotb against a simulator."""
+
+    release_install(session)
+
+    env = env_vars_for_test(sim, toplevel_lang, gpi_interface)
+    config_str = stringify_dict(env)
+
+    session.log(f"Running tests against a simulator: {config_str}")
+    session.run("make", "test", external=True, env=env)
+
+    session.log(f"Running simulator-specific tests against a simulator {config_str}")
+    session.run(
+        "pytest",
+        "-v",
+        "-k",
+        "simulator_required",
+    )
+
+    session.log(f"All tests passed with configuration {config_str}!")
+
+
+@nox.session
+def release_test_nosim(session: nox.Session) -> None:
+    """Run the simulator-agnostic tests against a cocotb release."""
+
+    release_install(session)
+
+    session.log("Running simulator-agnostic tests")
+    session.run(
+        "pytest",
+        "-v",
+        "-k",
+        "not simulator_required",
+    )
+
+    session.log("All tests passed!")
+
+
 @nox.session
 def docs(session: nox.Session) -> None:
     """invoke sphinx-build to build the HTML docs"""


Test patch: