Repository: dankamongmen/notcurses (language: python)

Issue:
become a thing on windows
In a Kantian sense, it would seem a good thing if notcurses worked on windows, for whatever definition of works that means. I very much doubt that I will ever undertake this work, but if some brave soul wishes to take a stab at it, or report problems, or whatever, here's a bug in which you might do that.
start running windows github actions
@igo95862 , @michaelsbradleyjr , i think this was probably in the various discussions, but:

* i'm seeing a windows workflow in `.github/workflows`, and
* https://dvdhrm.github.io/2021/04/21/meson-msvc-github-actions/ suggests i ought be able to build on windows at github's expense (and prepared infrastructure), yet
* our windows build doesn't appear to be running

what need i do to turn it on? i'm hoping to have windows and mac turned up soon.

this is a prerequisite for #463 


Discussion / hints:
https://github.com/microsoft/terminal would be our target, i am thinking
I am interested at taking a stab at this, cannot promise anything though

Any pointers?
> I am interested at taking a stab at this, cannot promise anything though
> Any pointers?

Neat! Feel free to hit me up for any assistance you might need. Unfortunately, I don't have much advice at this moment -- I haven't written code for Windows in over twenty years. Probably the first thing would be prepping our build infrastructure to run on a Windows machine with VS/VC++ --- I chose CMake largely because it supposedly makes Windows development easier.

I doubt very much that we'll build out of the gate, but once we have some kind of "I run this, expect this, but instead get this" loop, we ought be able to advance it quickly.

Next would be determining what the state of terminal support is on Windows. Is terminfo a meaningful concept there? What are the major terminals? In what ways are they similar and different from UNIX terminals?

So there are a few questions of basic groundbreaking that still need answers IMHO.
Since it's a console library, can I suggest that we make sure it builds under mingw-w64 as well? It's easy to set up and it would be nice to be able to build **and** run the tests on one machine (with WINE). CMake is shipped with VS2019, so yes, it would make stuff easier, however another possibility is to use clang on Windows since with that you don't need the whole IDE installed, just the Windows SDK. With CMake all of MSVC, mingw and clang can be supported. In Xamarin.Android we use cmake to build our Windows bits on Linux/macOS with mingw-w64, here are [32-bit](https://github.com/xamarin/xamarin-android/blob/4b433e8912a3068af18f8fde67fb210f30b91286/build-tools/scripts/mingw-32.cmake.in) and [64-bit](https://github.com/xamarin/xamarin-android/blob/4b433e8912a3068af18f8fde67fb210f30b91286/build-tools/scripts/mingw-64.cmake.in) CMake toolchain definitions for this.

Like Nick, I haven't coded for Windows for more than 20 years, so I don't know the state of the console, but I suspect ncurses is not a big thing there and so terminfo might be hard to support, however there's https://github.com/mauke/unibilium/ which provides the necessary terminfo support, they also mention a thing or two about Windows. @dankamongmen maybe it would be good to consider Unibilium as the ncurses replacement for notcurses? Last time I tried to build notcurses on macOS, it couldn't find libtinfo (neither the shipped nor the homebrew ncurses had it - they *may* had the stuff compiled into the main library, I didn't check tbh) and so if we used Unibilium we'd be better off on systems where ncurses isn't native or just not well supported.

Lastly, @sandorex you might find https://github.com/Microsoft/vcpkg useful for building stuff on Windows as far as dependencies are concerned, here's how our LibZipSharp [uses it](https://github.com/xamarin/LibZipSharp/blob/d321af1d2ac581bd0fd1e6a01f6c0b94ba1ee813/azure-pipelines.yml#L27-L41).
Some information I've gathered about current state of console on Windows (take it with a grain of salt)

Both the new and the old terminals should be treated as `xterm-256color` *(this may change in the future)* as said [here](https://github.com/microsoft/terminal/issues/2958#issuecomment-635318192)
> For all intents and purposes, the Windows Terminal should be treated as `xterm-256color`, and any bugs that might arise from that should be filed here, so we can close any remaining gaps.

Both terminals support truecolor (but ill have to test that)
@grendello It seems unibilium is unmaintained so it would be better to use the [neovim fork](https://github.com/neovim/unibilium)
@sandorex sounds good!
I've started making a unibilium replacement, its made in c++ but will have c wrappers, would that be acceptable to be used?

Other question i have is, are there any uses for ncurses other than getting the capabilities ?
Im adding extended terminfo support so it should be able to read anything ncurses can
Has anyone tried compiling notcurses with msys2/cygwin? If not, I can give it a go, and report my findings.
So, after attempting to compile it with msys2 and cygwin, it gets stuck at the same place OSX does, it can't find uchar.h. I feel like a workaround for this might be to add a compile-time option to disable unicode and use plain ASCII or use a more portable unicode library.
https://github.com/dankamongmen/notcurses/runs/1101716618?check_suite_focus=true hark! a github action appears!
when we do this, we'll want to package up in [vcpkg](https://github.com/microsoft/vcpkg)
Let me share my knowledge about Windows consoles with you. This is what I learned while working on https://github.com/magiblot/tvision.

> Is terminfo a meaningful concept there?

I would say it's not. https://docs.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences contains all you need to know.

> What are the major terminals? In what ways are they similar and different from UNIX terminals?

During many years Conhost was the only relevant console on Windows. In Windows, the Console is a feature provided by the operating system. Every process can be associated to one Console, which is an instance of Conhost. Until Conhost received support for virtual terminal sequences, native TUI applications could only interact with this console using the [Win32 Console API](https://docs.microsoft.com/en-us/windows/console/console-functions) instead of stdio. In order to create your own terminal emulator, you had to wrap it around Conhost in some inefficient way.

Windows did not have an equivalent of `forkpty` until 2018, when [the Pseudo Console API was introduced](https://devblogs.microsoft.com/commandline/windows-command-line-introducing-the-windows-pseudo-console-conpty/). This technology not only provides a PTY infrastructure, but is also a translation layer for client applications using the legacy Win32 Console API. Windows Terminal is an example of a terminal emulator relying on this API.

The above means that Windows developers have not had much time to write terminal emulators with diverging capabilites. Therefore, terminfo has little importance here.

Unix-like environments like MSYS2 or Cygwin may use terminfo because most applications running in them are ported straight from Unix systems. But terminfo won't be of any use if your application is running under Conhost unless you assume `TERM=xterm-256color`, like someone said earlier.

> So, after attempting to compile it with msys2 and cygwin, it gets stuck at the same place OSX does, it can't find uchar.h.

You won't face this issue in MinGW or VC++.

> Windows, sadly, loves UTF16. It is likely that we would need to handle UTF16 from Windows applications. We should support emitting UTF16 as output, perhaps.

UTF-16 support is certainly not necessary. You can interact with the console in UTF-8 as long as you [configure the console codepage properly](https://github.com/Embarcadero/Dev-Cpp/issues/103#issuecomment-747499859). Additionally, since 2018, [UTF-8 is also supported in the C Runtime](https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/setlocale-wsetlocale?view=msvc-160&viewFallbackFrom=vs-2019#utf-8-support). So you can write a fully capable Unicode application on Windows without a single wide char, and I find it a good idea to encourage users to do so. The problem in notcurses will be that all interfaces assuming `wchar_t` to be 32-bit (if there are any) will be ill-formed on Windows.

Another difference compared to Unix terminals are character widths. While double-width characters are supported as you would expect, combining characters are drawn as single-width characters. For example, in Windows Terminal, the sequence `में` (1 base character + 2 combining characters) is 3 columns wide, even though the glyph is rendered correctly.

Cheers.
recent events have me hoping that this will see life by 3.0.0 this fall
I believe it's because the GHA workflows for Windows and macOS depend on 3rd party actions. The settings for this repository would need to be changed to allow 3rd party actions to run as part of a workflow.

These are the 3rd party actions related to Windows:

(maintained by the same people who maintain MSYS2)
https://github.com/dankamongmen/notcurses/blob/master/.github/workflows/windows_test.yml#L17

(maintained by GitHub employees)
https://github.com/dankamongmen/notcurses/blob/master/.github/workflows/windows_test.yml#L34

And this is the one related to macOS:

(maintained by GitHub employees)
https://github.com/dankamongmen/notcurses/blob/master/.github/workflows/macos_test.yml#L28
sweet, let me go see what i can do, thanks man!
i changed "allow local actions only" to "allow all actions". let's see what happens!
They can be run now, but still need to be triggered manually, from the Actions menu.
I can remove the necessity for the Windows build to be manually triggered, if that is desirable.
I would keep it as manual until Windows works as intended. MacOS may be able to part of the automated CI now though.

Fix patch:
diff --git a/README.md b/README.md
index e933fba3b1..8ae34a9e68 100644
--- a/README.md
+++ b/README.md
@@ -52,6 +52,7 @@ sudo apt-get update
 [![Build](https://drone.dsscaw.com:4443/api/badges/dankamongmen/notcurses/status.svg)](https://drone.dsscaw.com:4443/dankamongmen/notcurses)
 [![Ubuntu](https://github.com/dankamongmen/notcurses/actions/workflows/ubuntu_test.yml/badge.svg?branch=master)](https://github.com/dankamongmen/notcurses/actions/workflows/ubuntu_test.yml?query=branch%3Amaster)
 [![macOS](https://github.com/dankamongmen/notcurses/actions/workflows/macos_test.yml/badge.svg?branch=master)](https://github.com/dankamongmen/notcurses/actions/workflows/macos_test.yml?query=branch%3Amaster)
+[![Windows](https://github.com/dankamongmen/notcurses/actions/workflows/windows_test.yml/badge.svg?branch=master)](https://github.com/dankamongmen/notcurses/actions/workflows/windows_test.yml?query=branch%3Amaster)
 
 [![pypi_version](https://img.shields.io/pypi/v/notcurses?label=pypi)](https://pypi.org/project/notcurses)
 [![crates.io](https://img.shields.io/crates/v/libnotcurses-sys.svg)](https://crates.io/crates/libnotcurses-sys)
diff --git a/include/notcurses/notcurses.h b/include/notcurses/notcurses.h
index f3fbebdf51..517f4113a6 100644
--- a/include/notcurses/notcurses.h
+++ b/include/notcurses/notcurses.h
@@ -20,6 +20,9 @@
 #elif defined(__APPLE__)
 #include <libkern/OSByteOrder.h>
 #define htole(x) (OSSwapInt32(htonl(x)))
+#elif defined(__MINGW64__)
+#include <winsock.h>
+#define htole(x) (bswap32(htonl(x)))
 #else
 #include <sys/endian.h>
 #define htole(x) (bswap32(htonl(x)))


Test patch:
diff --git a/.github/workflows/windows_test.yml b/.github/workflows/windows_test.yml
index be9f87e7f6..7cb3e75405 100644
--- a/.github/workflows/windows_test.yml
+++ b/.github/workflows/windows_test.yml
@@ -2,52 +2,55 @@
 name: Windows
 
 on:
-  workflow_dispatch:
+  pull_request:
+  push:
+    branches:
+      - master
 
 jobs:
   tests:
+    defaults:
+      run:
+        shell: msys2 {0}
     env:
       COLORTERM: truecolor
       NPROC: 2
       TERM: xterm
     name: 🏁 build, test, & install
     runs-on: windows-latest
+
     steps:
 
       - name: Install tools and libraries via MSYS2
         uses: msys2/setup-msys2@v2
         with:
-          msystem: MINGW64
+          msystem: UCRT64
           update: true
-          # installing mingw-w64-x86_64-libxml2 is a workaround for:
-          # https://github.com/msys2/MINGW-packages/issues/8658
           install: >
             base-devel
             git
-            mingw-w64-x86_64-cmake
-            mingw-w64-x86_64-ffmpeg
-            mingw-w64-x86_64-libunistring
-            mingw-w64-x86_64-libxml2
-            mingw-w64-x86_64-ncurses
-            mingw-w64-x86_64-rust
-            mingw-w64-x86_64-toolchain
+            mingw-w64-ucrt-x86_64-cmake
+            mingw-w64-ucrt-x86_64-ffmpeg
+            mingw-w64-ucrt-x86_64-libunistring
+            mingw-w64-ucrt-x86_64-ncurses
+            mingw-w64-ucrt-x86_64-rust
+            mingw-w64-ucrt-x86_64-toolchain
 
       - uses: actions/checkout@v2
 
-      - name: cmake
+      - name: Build libqrcodegen
         run: |
-          pushd .
           mkdir "${HOME}/repos" && cd "${HOME}/repos"
           git clone https://github.com/nayuki/QR-Code-generator.git
           cd QR-Code-generator/c
-          AR="$(which ar)"
-          make AR="${AR}"
-          PREFIX=/mingw64
-          cd ${PREFIX}/include && \
-            ln -s "${HOME}/repos/QR-Code-generator/c" qrcodegen
-          cd ${PREFIX}/lib && \
-            ln -s "${HOME}/repos/QR-Code-generator/c/libqrcodegen.a" .
-          popd
+          make
+          cd /ucrt64/include && \
+            ln -s "${HOME}/repos/QR-Code-generator/c" ./qrcodegen
+          cd /ucrt64/lib && \
+            ln -s "${HOME}/repos/QR-Code-generator/c/libqrcodegen.a" ./libqrcodegen.a
+
+      - name: cmake
+        run: |
           mkdir build && cd build
           cmake .. \
             -G "MSYS Makefiles" \
@@ -69,8 +72,7 @@ jobs:
       - name: make install
         run: |
           cd build
-          sudo make install
-          sudo ldconfig
+          make install
 
       - name: python wrappers
         run: |
@@ -78,7 +80,7 @@ jobs:
           pip install pypandoc
           cd cffi
           python3 setup.py sdist build
-          sudo python3 setup.py install
+          python3 setup.py install
           notcurses-pydemo > /dev/null
           ncdirect-pydemo > /dev/null