fish-shell/fish-shell — pull request #1271 by g-nix (2014-01-27 23:35:52+00:00)
Title: Experimental option: build and link static ncurses with builtin terminfo (no system dep!)

So I spent some time (too much time, like a week) finally figuring out how to build ncurses such that it is completely independent from external terminfo, and can be linked like any normal static library.

Short version:
This pull adds a configure flag `--with-builtin-terminfo` which calls a script in build_tools which in turn downloads, configures, and sets this up for fish in one go. There should be no other functionality change. A more lengthy rationale follows.

Background/refresher:
I realize most of you will already know all this, but I thought it might be worth recapping. Fish is _super pretty_ and can do things like fast auto-systax-highlighting because unlike other shells, fish draws to the terminal (exclusively?) by way of [n]curses library, which basically provides an abstraction over "what heinous escape sequence is needed to do some crazy thing like move the cursor arbitrarily, switch colors, etc." Curses is a UNIX (though not POSIX) standard, although the standard is quite limited. There used to be a few different curses implementations, but today I think only ncurses, NetBSD curses, and illumos libxcurses survive. (Does fish even build against these today? There's some code in configure.in, etc, but I'm not sure?)

In any case, ncurses is more or less the standard today. It's a good library, but it's kinda funky since the codebase is so old. It (typically) works by reading a terminfo "database" (a directory structure, usually at `/usr/share/terminfo`, where a bunch of terminal capability files are kept), which correspond to what's set in the $TERM variable.

The problem with that approach is that even when a new version of the library is built statically (with a typical install), the location of the terminfo directory is hardcoded. Therefore, if this directory is moved, modified or corrupted, fish built like this will simply die. So the system library/terminfo is almost always used.

As it turns out, ncurses _can_ in fact be configured to include terminfo in the library itself. Why no one does this, I'm not sure... though I expect it has something to do with the fact the when built from source, ncurses has some ~120 custom configure options, most of which change the ABI, and its not clear at all which ones do what you want. I'm almost positive, in fact, that _no one_ is doing this right now, since in the process I discovered a bug that breaks this configuration that was introduced last August (sent upstream a minute ago). The script includes a small patch for now, and just emits a warning if the patch fails.

Nonetheless, I think there are a number of reasons to consider this:
- Relying on system ncurses and terminfo sucks, because its often old or buggy. I.e., Mac OS X 10.9 ships with ncurses 5.7 circa 2008.
- Relying on system ncurses and terminfo is slow: it has to dlopen the shared library, which in turn opens the terminfo file for the application with each invocation.
- Its easy to mess up the system config, as mentioned above, or by accidentally installing a bad file to `~/.terminfo`.
- This seems much faster, on my machine at least. And there are fewer jitters and artifacts.
- When fish goes fully multithreaded/reentrant, it's probably going to need to use a threaded curses lib as well (or no ...?), and it's not clear one can rely on the system lib to have been configured that way.
- Allows for more filesystem portability and resiliency against system weirdness.
- Etc., etc. Add your own.

Cons:
- This current first implementation approximately doubles the linked binary size. If it seems even larger, try stripping your binaries. This is only because I've included basically every usable terminfo entry here — with some investigation, one probably cut this down to ~10% of that.
- Etc., etc. Add your own.

So, um, yeah. Let me know how this works for you? Other thoughts/considerations?