Repository: dankamongmen/notcurses (language: python)

Issue:
rigorous scrolling
from notes in #2378...

there are really only four cases:

* alternate screen: no need to scroll, nor position cursor. when we exit, we're placed back exactly where we ought be, and the content we've drawn is hidden.
* no alternate screen, no scrolling: place cursor at the furthest point to which we've written, never scroll
* no alt, scrolling, case 1: we write to the standard plane at either the last cell, or placing a newline on the last line. this forces a (physical) scroll. emit the actual newline for the latter case; in the former case, just ride.
* no alt, scrolling, case 2: we write to some other plane n lines beyond the bottom of the standard plane. maybe we want a scroll, maybe we don't. it must be indicated by a client call to scroll_to_plane() or whatnot. this is effected by executing n physical scrolls prior to drawing.
in both of these last two cases, the cursor ought always be finally placed on the last line, but never causing a scroll in so doing. scrolling happens before drawing. cursor finalization happens after, and never the twain shall meet.

for one, we seem to be misprinting the first line of the infoplane in `ncneofetch` in `xfce4-terminal`. but i want a unified, rigorous approach to scrolling that the user doesn't have to think about. finally.
rigorous scrolling
from notes in #2378...

there are really only four cases:

* alternate screen: no need to scroll, nor position cursor. when we exit, we're placed back exactly where we ought be, and the content we've drawn is hidden.
* no alternate screen, no scrolling: place cursor at the furthest point to which we've written, never scroll
* no alt, scrolling, case 1: we write to the standard plane at either the last cell, or placing a newline on the last line. this forces a (physical) scroll. emit the actual newline for the latter case; in the former case, just ride.
* no alt, scrolling, case 2: we write to some other plane n lines beyond the bottom of the standard plane. maybe we want a scroll, maybe we don't. it must be indicated by a client call to scroll_to_plane() or whatnot. this is effected by executing n physical scrolls prior to drawing.
in both of these last two cases, the cursor ought always be finally placed on the last line, but never causing a scroll in so doing. scrolling happens before drawing. cursor finalization happens after, and never the twain shall meet.

for one, we seem to be misprinting the first line of the infoplane in `ncneofetch` in `xfce4-terminal`. but i want a unified, rigorous approach to scrolling that the user doesn't have to think about. finally.


Discussion / hints:
one thing to look at (see #2026) is when we've got some sprixel plane, either at the bottom or not
this is pretty much going to define 4.0, i think
i'd really like to get this resolved for 3.0.7 and in the next ubuntu LTS.
so the most fundamental thing is, i think:

**when the standard plane is scrolling, a scroll event there ought result in a real-world scroll, and this is the only thing which ought ever result in a real-world scroll**

now the problem of course is that we might have all kinds of scrolling going on without a call to `notcurses_render()`, and thus they're lost. i only see two possible ways around this:

* we call `notcurses_render()` on a standard plane scroll event, with whatever we have at the time. brings up all kinds of thread safety issues -- you basically can't do any other actions concurrently in a pile containing the standard plane if the standard plane might scroll.
* make it an error to write output that would cause the standard plane to scroll, forcing the user to instead call `notcurses_render()`

i think the first option is the one we need to go with. this kinda drastically changes the model, but it's also the only real way to make it so that "you can dump output to the standard plane, and don't need to worry about rendering on scroll". of course, it also means you have to be careful -- a scroll event can dump output before you expect -- and implies the threading issues mentioned above.

how will this work with existing code? in the best case, we'll just have some extraneous calls to `notcurses_render()`. in the worst case, i guess it'll break it -- we'll see things we weren't supposed to, etc.
what about planes placed below the standard plane? i don't think they would result in a scroll unless `scroll_parent_to_child()` or whatever it is is called, in which case that would operate under the same rules as general output to the (scrolling) standard plane -- a scroll there results in a real-world scroll.

how need we update the internal rendering model for this? i think maybe not at all -- the existing infrastructure for scrolling in a plane handles it?
let's see if we can't knock out #2557 as an effect of doing this
i guess we know this is beginning to work when we can pull the `notcurses_render()` out of e.g. `src/poc/scrolling.c` and it continues to work? hrmmm, no...that raises an interesting point. we won't do the implicit render until we hit a scrolling event. so if they want output before such an event, they still *do* need to explicitly render...well, that's reasonable, i suppose.
one thing to look at (see #2026) is when we've got some sprixel plane, either at the bottom or not
this is pretty much going to define 4.0, i think
i'd really like to get this resolved for 3.0.7 and in the next ubuntu LTS.
so the most fundamental thing is, i think:

**when the standard plane is scrolling, a scroll event there ought result in a real-world scroll, and this is the only thing which ought ever result in a real-world scroll**

now the problem of course is that we might have all kinds of scrolling going on without a call to `notcurses_render()`, and thus they're lost. i only see two possible ways around this:

* we call `notcurses_render()` on a standard plane scroll event, with whatever we have at the time. brings up all kinds of thread safety issues -- you basically can't do any other actions concurrently in a pile containing the standard plane if the standard plane might scroll.
* make it an error to write output that would cause the standard plane to scroll, forcing the user to instead call `notcurses_render()`

i think the first option is the one we need to go with. this kinda drastically changes the model, but it's also the only real way to make it so that "you can dump output to the standard plane, and don't need to worry about rendering on scroll". of course, it also means you have to be careful -- a scroll event can dump output before you expect -- and implies the threading issues mentioned above.

how will this work with existing code? in the best case, we'll just have some extraneous calls to `notcurses_render()`. in the worst case, i guess it'll break it -- we'll see things we weren't supposed to, etc.
what about planes placed below the standard plane? i don't think they would result in a scroll unless `scroll_parent_to_child()` or whatever it is is called, in which case that would operate under the same rules as general output to the (scrolling) standard plane -- a scroll there results in a real-world scroll.

how need we update the internal rendering model for this? i think maybe not at all -- the existing infrastructure for scrolling in a plane handles it?
let's see if we can't knock out #2557 as an effect of doing this
i guess we know this is beginning to work when we can pull the `notcurses_render()` out of e.g. `src/poc/scrolling.c` and it continues to work? hrmmm, no...that raises an interesting point. we won't do the implicit render until we hit a scrolling event. so if they want output before such an event, they still *do* need to explicitly render...well, that's reasonable, i suppose.

Fix patch:
diff --git a/src/lib/notcurses.c b/src/lib/notcurses.c
index de675e0d8..b8fd66d21 100644
--- a/src/lib/notcurses.c
+++ b/src/lib/notcurses.c
@@ -1728,11 +1728,15 @@ void scroll_down(ncplane* n){
 //fprintf(stderr, "pre-scroll: %d/%d %d/%d log: %d scrolling: %u\n", n->y, n->x, n->leny, n->lenx, n->logrow, n->scrolling);
   n->x = 0;
   if(n->y == n->leny - 1){
+    // we're on the last line of the plane
     if(n->autogrow){
       ncplane_resize_simple(n, n->leny + 1, n->lenx);
       ncplane_cursor_move_yx(n, n->leny - 1, 0);
       return;
     }
+    // we'll actually be scrolling material up and out, and making a new line.
+    // if this is the standard plane, that means a "physical" scroll event is
+    // called for.
     if(n == notcurses_stdplane(ncplane_notcurses(n))){
       ncplane_pile(n)->scrolls++;
     }
@@ -1843,9 +1847,11 @@ ncplane_put(ncplane* n, int y, int x, const char* egc, int cols,
       linesend = true;
     }
   }
+  bool scrolled = false;
   if(linesend){
     if(n->scrolling){
       scroll_down(n);
+      scrolled = true;
     }else if(n->autogrow){
       ncplane_resize_simple(n, n->leny, n->lenx + cols);
     }else{
@@ -1859,6 +1865,7 @@ ncplane_put(ncplane* n, int y, int x, const char* egc, int cols,
   }
   if(*egc == '\n'){
     scroll_down(n);
+    scrolled = true;
   }
   // A wide character obliterates anything to its immediate right (and marks
   // that cell as wide). Any character placed atop one cell of a wide character
@@ -1928,6 +1935,11 @@ ncplane_put(ncplane* n, int y, int x, const char* egc, int cols,
       ++n->x;
     }
   }
+  if(scrolled){
+    if(n == notcurses_stdplane(ncplane_notcurses(n))){
+      notcurses_render(ncplane_notcurses(n));
+    }
+  }
   return cols;
 }
 
diff --git a/src/lib/render.c b/src/lib/render.c
index c52659d24..cec6703e1 100644
--- a/src/lib/render.c
+++ b/src/lib/render.c
@@ -943,7 +943,9 @@ clean_sprixels(notcurses* nc, ncpile* p, fbuf* f, int scrolls){
   return bytesemitted;
 }
 
-// scroll the lastframe data |rows| up, to reflect scrolling reality
+// scroll the lastframe data |rows| up, to reflect scrolling reality.
+// FIXME we could virtualize this as we do for scrolling planes; this
+// method involves a lot of unnecessary copying.
 static void
 scroll_lastframe(notcurses* nc, unsigned rows){
   // the top |rows| rows need be released (though not more than the actual
diff --git a/src/lib/sixel.c b/src/lib/sixel.c
index 53b56f9f7..a01dffa6d 100644
--- a/src/lib/sixel.c
+++ b/src/lib/sixel.c
@@ -1434,7 +1434,7 @@ int sixel_blit(ncplane* n, int linesize, const void* data, int leny, int lenx,
   }
   scrub_color_table(bargs->u.pixel.spx);
   // we haven't actually emitted the body of the sixel yet. instead, we'll emit
-  // it at sixel_redraw(), thus avoided a double emission in the case of wipes
+  // it at sixel_redraw(), thus avoiding a double emission in the case of wipes
   // taking place before it's visible.
   bargs->u.pixel.spx->wipes_outstanding = 1;
   return r;
diff --git a/src/poc/cjkscroll.c b/src/poc/cjkscroll.c
index cc2fe8bb3..d5bfccb90 100644
--- a/src/poc/cjkscroll.c
+++ b/src/poc/cjkscroll.c
@@ -19,6 +19,7 @@ int main(void){
   ncplane_set_styles(n, NCSTYLE_BOLD);
   ncplane_putstr(n, "This program is *not* indicative of real scrolling speed.\n");
   ncplane_set_styles(n, NCSTYLE_NONE);
+  unsigned y = ncplane_cursor_y(n);
   while(true){
     struct timespec req = { .tv_sec = 0, .tv_nsec = 1000000, };
     nanosleep(&req, NULL);
@@ -28,10 +29,16 @@ int main(void){
     if(++wc == 0x9fa5){
       wc = 0x4e00;
     }
-    if(notcurses_render(nc)){
-      break;
+    unsigned newy = ncplane_cursor_y(n);
+    if(newy != y){
+      y = newy;
+      notcurses_render(nc);
     }
   }
+  if(notcurses_render(nc)){
+    notcurses_stop(nc);
+    return EXIT_FAILURE;
+  }
   notcurses_stop(nc);
   return EXIT_FAILURE;
 }
diff --git a/src/poc/quantanal.c b/src/poc/quantanal.c
deleted file mode 100644
index 0590875cf..000000000
--- a/src/poc/quantanal.c
+++ /dev/null
@@ -1,184 +0,0 @@
-#include <inttypes.h>
-#include <notcurses/notcurses.h>
-#include <compat/compat.h>
-
-static char*
-colorbuf(void){
-  return calloc(1u << 24u, 1);
-}
-
-static uint32_t
-count_colors(uint32_t pixel, char* buf, uint32_t colors){
-  if(buf[pixel & 0xffffffu] == 0){
-    buf[pixel & 0xffffffu] = 1;
-    return colors + 1;
-  }
-  return colors;
-}
-
-static int
-compare(const struct ncvisual* n1, const struct ncvisual* n2,
-        const ncvgeom* geom, struct notcurses* nc){
-  uint32_t co0 = 0, co1 = 0;
-  char* cbuf0 = colorbuf();
-  char* cbuf1 = colorbuf();
-  struct ncplane* stdn = notcurses_stdplane(nc);
-  uint64_t rdelta = 0;
-  uint64_t gdelta = 0;
-  uint64_t bdelta = 0;
-  unsigned ly = geom->rpixy;
-  unsigned lx = geom->rpixx;
-  ncplane_set_fg_rgb(stdn, 0x03c04a);
-  double p = 0;
-  for(unsigned y = 0 ; y < ly ; ++y){
-    for(unsigned x = 0 ; x < lx ; ++x){
-      uint32_t p0, p1;
-      if(ncvisual_at_yx(n1, y, x, &p0) ||
-         ncvisual_at_yx(n2, y, x, &p1)){
-        continue;
-      }
-      if(ncpixel_a(p0) < 192 || ncpixel_a(p1) < 192){
-        continue;
-      }
-      co0 = count_colors(p0, cbuf0, co0);
-      co1 = count_colors(p1, cbuf1, co1);
-      // three component differences for current pixel
-      int rd, gd, bd;
-      rd = ncpixel_r(p0) - ncpixel_r(p1);
-      gd = ncpixel_g(p0) - ncpixel_g(p1);
-      bd = ncpixel_b(p0) - ncpixel_b(p1);
-      if(rd < 0){
-        rd = -rd;
-      }
-      if(gd < 0){
-        gd = -gd;
-      }
-      if(bd < 0){
-        bd = -bd;
-      }
-      rdelta += rd;
-      gdelta += gd;
-      bdelta += bd;
-      ++p;
-//fprintf(stderr, "0x%08x 0x%08x\n", p0, p1);
-    }
-    ncplane_printf_yx(stdn, -1, 1, "%08u pixels analyzed", (y + 1) * lx);
-    ncplane_printf(stdn, " Δr %"PRIu64" Δg %"PRIu64" Δb %"PRIu64,
-                  rdelta, gdelta, bdelta);
-    notcurses_render(nc);
-  }
-  ncplane_putchar(stdn, '\n');
-  ncplane_printf(stdn, " Colors: %"PRIu32" vs %"PRIu32"\n", co0, co1);
-  ncplane_printf(stdn, " %.0fpx Δr %"PRIu64" (%.03g) Δg %"PRIu64" (%.03g) Δb %"PRIu64 " (%.03g)\n",
-                 p, rdelta, rdelta / p, gdelta, gdelta / p, bdelta, bdelta / p);
-  ncplane_printf(stdn, " avg diff per pixel: %.03g\n", (rdelta + gdelta + bdelta) / p);
-  notcurses_render(nc);
-  free(cbuf0);
-  free(cbuf1);
-  return 0;
-}
-
-int main(int argc, char** argv){
-  if(argc < 2){
-    fprintf(stderr, "usage: %s images..." NL, *argv);
-    return EXIT_FAILURE;
-  }
-  struct notcurses_options opts = {0};
-  opts.flags = NCOPTION_CLI_MODE
-               | NCOPTION_DRAIN_INPUT
-               | NCOPTION_SUPPRESS_BANNERS;
-  struct notcurses* nc = notcurses_init(&opts, NULL);
-  if(nc == NULL){
-    return EXIT_FAILURE;
-  }
-  ncpixelimpl_e blit = notcurses_check_pixel_support(nc);
-  if(blit != NCPIXEL_SIXEL){
-    notcurses_stop(nc);
-    fprintf(stderr, "needed pixel blit type %d (sixel), got %d\n",
-                    NCPIXEL_SIXEL, blit);
-    return EXIT_FAILURE;
-  }
-  struct ncplane* stdn = notcurses_stdplane(nc);
-  while(*++argv){
-    struct timespec start;
-    clock_gettime(CLOCK_MONOTONIC, &start);
-    ncplane_set_fg_rgb(stdn, 0xf9d71c);
-    ncplane_printf(stdn, "analyzing %s...\n", *argv);
-    notcurses_render(nc);
-    struct ncvisual* ncv = ncvisual_from_file(*argv);
-    if(ncv == NULL){
-      notcurses_stop(nc);
-      fprintf(stderr, "error opening %s" NL, *argv);
-      return EXIT_FAILURE;
-    }
-    struct ncplane* ncp = ncplane_dup(stdn, NULL);
-    if(ncp == NULL){
-      notcurses_stop(nc);
-      return EXIT_FAILURE;
-    }
-    struct ncvisual_options vopts = {0};
-    vopts.n = ncp;
-    vopts.blitter = NCBLIT_PIXEL;
-    vopts.flags = NCVISUAL_OPTION_NODEGRADE;
-    struct ncvgeom geom;
-    if(ncvisual_geom(nc, ncv, &vopts, &geom)){
-      ncplane_set_fg_rgb(stdn, 0xd16002);
-      ncplane_printf(stdn, " Image too large, scaling to display\n");
-      notcurses_render(nc);
-      vopts.scaling = NCSCALE_STRETCH;
-      if(ncvisual_geom(nc, ncv, &vopts, &geom)){
-        notcurses_stop(nc);
-        fprintf(stderr, "error geometrizing %s" NL, *argv);
-        return EXIT_FAILURE;
-      }
-    }
-    ncplane_set_fg_rgb(stdn, 0x5dbb63);
-    unsigned rgbabytes = 4 * geom.rpixy * geom.rpixx;
-    ncplane_printf(stdn, " source pixels: %ux%u rendered: %ux%u %uB\n",
-                   geom.pixy, geom.pixx, geom.rpixy, geom.rpixx, rgbabytes);
-    notcurses_render(nc);
-    if(ncvisual_blit(nc, ncv, &vopts) == NULL){
-      notcurses_stop(nc);
-      fprintf(stderr, "error rendering %s" NL, *argv);
-      return EXIT_FAILURE;
-    }
-    char* s;
-    if((s = ncplane_at_yx(ncp, 0, 0, NULL, NULL)) == NULL){
-      notcurses_stop(nc);
-      fprintf(stderr, "error retrieving sixel for %s" NL, *argv);
-      return EXIT_FAILURE;
-    }
-    ncplane_set_fg_rgb(stdn, 0x74b72e);
-    size_t slen = strlen(s);
-    ncplane_printf(stdn, " control sequence: %" PRIuPTR " byte%s (%.4g%%)\n",
-                   slen, slen == 1 ? "" : "s", (double)slen * 100 / rgbabytes);
-    ncplane_reparent(ncp, ncp);
-    notcurses_render(nc);
-    unsigned leny = geom.rpixy;
-    unsigned lenx = geom.rpixx;
-    struct ncvisual* quantncv = ncvisual_from_sixel(s, leny, lenx);
-    if(quantncv == NULL){
-      notcurses_stop(nc);
-      fprintf(stderr, "error loading %" PRIuPTR "B sixel" NL, slen);
-      free(s);
-      return EXIT_FAILURE;
-    }
-    free(s);
-    if(compare(ncv, quantncv, &geom, nc)){
-      notcurses_stop(nc);
-      fprintf(stderr, "error comparing sixels" NL);
-      return EXIT_FAILURE;
-    }
-    ncvisual_destroy(quantncv);
-    ncvisual_destroy(ncv);
-    struct timespec end;
-    clock_gettime(CLOCK_MONOTONIC, &end);
-    uint64_t t0 = timespec_to_ns(&start);
-    uint64_t t1 = timespec_to_ns(&end);
-    ncplane_set_fg_rgb(stdn, 0x03ac13);
-    ncplane_printf(stdn, "done with %s in %.3fms.\n", *argv, (t1 - t0) / 1000000.0);
-    ncplane_destroy(ncp);
-    notcurses_render(nc);
-  }
-  return notcurses_stop(nc) ? EXIT_FAILURE : EXIT_SUCCESS;
-}
diff --git a/src/poc/scroll.c b/src/poc/scroll.c
index e45ce56b9..2147bfc10 100644
--- a/src/poc/scroll.c
+++ b/src/poc/scroll.c
@@ -19,6 +19,7 @@ int main(void){
   ncplane_set_styles(n, NCSTYLE_BOLD);
   ncplane_putstr(n, "This program is *not* indicative of real scrolling speed.\n");
   ncplane_set_styles(n, NCSTYLE_NONE);
+  unsigned y = ncplane_cursor_y(n);
   while(true){
     struct timespec req = { .tv_sec = 0, .tv_nsec = 1000000, };
     nanosleep(&req, NULL);
@@ -28,10 +29,16 @@ int main(void){
     if(++c == '{'){
       c = 'A';
     }
-    if(notcurses_render(nc)){
-      break;
+    unsigned newy = ncplane_cursor_y(n);
+    if(newy != y){
+      y = newy;
+      notcurses_render(nc);
     }
   }
+  if(notcurses_render(nc)){
+    notcurses_stop(nc);
+    return EXIT_FAILURE;
+  }
   notcurses_stop(nc);
   return EXIT_SUCCESS;
 }


Test patch:
diff --git a/src/tests/sixels.cpp b/src/tests/sixels.cpp
index 29d9e4554..4f840a0a8 100644
--- a/src/tests/sixels.cpp
+++ b/src/tests/sixels.cpp
@@ -22,7 +22,17 @@ TEST_CASE("Sixels") {
   auto n_ = notcurses_stdplane(nc_);
   REQUIRE(n_);
 
-  // this can only run with a Sixel backend
+  // exercise ncvisual_from_sixel()
+  SUBCASE("LoadSixel") {
+    auto ncv = ncvisual_from_sixel("\x1bP0;1;0q\"1;1;1;6#0;2;66;18;80#0@-\x1b\\", 1, 1);
+    REQUIRE(ncv);
+    uint32_t p;
+    ncvisual_at_yx(ncv, 0, 0, &p);
+    CHECK(0xffcc2da8 == p);
+    ncvisual_destroy(ncv);
+  }
+
+  // remaining tests can only run with a Sixel backend
   if(notcurses_check_pixel_support(nc_) <= 0){
     CHECK(0 == notcurses_stop(nc_));
     return;