Repository: flekschas/regl-scatterplot (js)

Issue:
Point circles look malformed
I think the circles look a bit off. Take a look at these (8x zoom):

![image](https://github.com/flekschas/regl-scatterplot/assets/199380/6ca66efb-c763-4e92-b79f-e964c87fd692)

They are not particularily round and look a bit egg shaped. I have tinkered with the code and I have two ideas for improvement. 

1. Increase the anti-aliasing from 1px to 3px:

![image](https://github.com/flekschas/regl-scatterplot/assets/199380/bd6d033a-4ab1-44e1-859b-52eab44a04ce)

The change is in the fragment shader `point.fs`:
```diff
-  float alpha = linearstep(finalPointSize + 0.5, finalPointSize - 0.5, sdf);
+  float alpha = linearstep(finalPointSize + 0.5, finalPointSize - 2.5, sdf);
```

2. Snap points to pixels. This makes them perfectly symmetrical, like this:

![image](https://github.com/flekschas/regl-scatterplot/assets/199380/9f609d3c-f68e-42d7-a4d2-9e86f3dbe93a)

The change is in the vertex shader `point.vs`, and you have to provide the width and height in `index.js`.

```diff  
-  gl_Position = modelViewProjection * vec4(state.x, state.y, 0.0, 1.0);
+  vec4 clipSpacePosition = modelViewProjection * vec4(state.x, state.y, 0.0, 1.0);
+  vec2 ndcPosition = clipSpacePosition.xy / clipSpacePosition.w;
+  vec2 pixelPos = 0.5 * (ndcPosition + 1.0) * vec2(drawingBufferWidth, drawingBufferHeight);
+  pixelPos = floor(pixelPos + 0.5); // Snap to nearest pixel
+  vec2 snappedPosition = (pixelPos / vec2(drawingBufferWidth, drawingBufferHeight)) * 2.0 - 1.0;
+  gl_Position = vec4(snappedPosition, 0.0, 1.0);
```

```diff
+       drawingBufferWidth: (context) => context.drawingBufferWidth,
+       drawingBufferHeight: (context) => context.drawingBufferHeight,


```

These two can be combined, of course. Then it looks like this:

![image](https://github.com/flekschas/regl-scatterplot/assets/199380/83a4b4fe-3ea1-4f37-927a-431e2b256626)

The downside with pixel-snapped coordinates is that between two zoom levels only some of the points are affected,  making it look like they jump (if you look very carefully, but looking carefully is what this issue is about). In this video you can for example focus on the two overlapping points and you see that they don't wiggle in unison:

https://github.com/flekschas/regl-scatterplot/assets/199380/f5fa8883-ad77-4aa5-bb93-cd23607ec2cc

Due to the zoom wiggling with pixel-snapping I think the most straight-forward way to improve the circles is to increase the anti-aliasing. Happy to hear your opinion on this!

Pull request:
feat: improve export functionality
This PR improves `scatterplot.export()` by allowing to customize the rendering settings during the export.

## Description

> What was changed in this pull request?

- Added `antiAliasing` property to allow adjusting the anti-aliasing of points
- Added `pixelAligned` property to allow aligning points with the pixel grid
- Enhanced `export()` to allow up or down scaling the exported image as well as adjusting the anti-aliasing and pixel alignment settings during export
- Exposed the renderer's `resize()` function in case one wants to resize the renderer's canvas.

> Why is it necessary?

Fixes #175

All changes enable better exporting. E.g., during export it can be useful to upscale the image, render the points a tad smoother by increasing the anti-aliasing and snapping points to the pixel grid.

Huge thanks go to @danr for providing the necessary code changes in #175! 🙏 

## Checklist

- [x] Provided a concise title as a [semantic commit message](https://www.conventionalcommits.org) (e.g. "fix: correctly handle undefined properties")
- [x] `CHANGELOG.md` updated
- [x] Tests added or updated
- [x] Documentation in `README.md` added or updated
- [ ] Example(s) added or updated
- [ ] Screenshot, gif, or video attached for visual changes


Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index a52d379..f820938 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,10 @@
+## 1.12.0
+
+- Feat: add support for adjusting the anti-aliasing via `scatterplot.set({ antiAliasing: 1 })`. ([#175](https://github.com/flekschas/regl-scatterplot/issues/175))
+- Feat: add support for aligning points with the pixel grid via `scatterplot.set({ pixelAligned: true })`. ([#175](https://github.com/flekschas/regl-scatterplot/issues/175))
+- Feat: enhance `scatterplot.export()` by allowing to adjust the scale, anti-aliasing, and pixel alignment. Note that when customizing the render setting for export, the function returns a promise that resolves into `ImageData`.
+- Feat: expose `resize()` method of the `renderer`.
+
 ## 1.11.4
 
 - Fix: allow setting the lasso long press indicator parent element
diff --git a/README.md b/README.md
index 9d19ee3..73630c4 100644
--- a/README.md
+++ b/README.md
@@ -707,9 +707,12 @@ Sets the view back to the initially defined view. This will trigger a `view` eve
 
 **Arguments:**
 
-- `options` is an object for customizing how to export. See [regl.read()](https://github.com/regl-project/regl/blob/master/API.md#reading-pixels) for details.
+- `options` is an object for customizing the render settings during the export:
+  - `scale`: is a float number allowning to adjust the exported image size
+  - `antiAliasing`: is a float allowing to adjust the anti-aliasing factor
+  - `pixelAligned`: is a Boolean allowing to adjust the point alignment with the pixel grid
 
-**Returns:** an object with three properties: `pixels`, `width`, and `height`. The `pixels` is a `Uint8ClampedArray`.
+**Returns:** an [`ImageData`](https://developer.mozilla.org/en-US/docs/Web/API/ImageData) object if `option` is `undefined`. Otherwise it returns a Promise resolving to an [`ImageData`](https://developer.mozilla.org/en-US/docs/Web/API/ImageData) object.
 
 <a name="scatterplot.subscribe" href="#scatterplot.subscribe">#</a> scatterplot.<b>subscribe</b>(<i>eventName</i>, <i>eventHandler</i>)
 
@@ -818,6 +821,8 @@ can be read and written via [`scatterplot.get()`](#scatterplot.get) and [`scatte
 | annotationLineColor                   | string or quadruple                          | `[1, 1, 1, 0.1]`                    | hex, rgb, rgba                                                  | `true`   | `false`     |
 | annotationLineWidth                   | number                                       | `1`                                 |                                                                 | `true`   | `false`     |
 | annotationHVLineLimit                 | number                                       | `1000`                              | the extent of horizontal or vertical lines                      | `true`   | `false`     |
+| antiAliasing                          | number                                       | `0.5`                               | higher values result in more blurry points                      | `true`   | `false`     |
+| pixelAligned                          | number                                       | `false`                             | if true, points are aligned with the pixel grid                 | `true`   | `false`     |
 
 <a name="property-notes" href="#property-notes">#</a> <b>Notes:</b>
 
diff --git a/src/constants.js b/src/constants.js
index 5f004f5..fe3e1fc 100644
--- a/src/constants.js
+++ b/src/constants.js
@@ -167,6 +167,8 @@ export const W_NAMES = new Set(['w', 'valueW', 'valueB', 'value2', 'value']);
 export const DEFAULT_IMAGE_LOAD_TIMEOUT = 15000;
 export const DEFAULT_SPATIAL_INDEX_USE_WORKER = undefined;
 export const DEFAULT_CAMERA_IS_FIXED = false;
+export const DEFAULT_ANTI_ALIASING = 0.5;
+export const DEFAULT_PIXEL_ALIGNED = false;
 
 // Error messages
 export const ERROR_POINTS_NOT_DRAWN = 'Points have not been drawn';
diff --git a/src/index.js b/src/index.js
index 6788cff..228e30d 100644
--- a/src/index.js
+++ b/src/index.js
@@ -37,6 +37,7 @@ import {
   DEFAULT_ANNOTATION_HVLINE_LIMIT,
   DEFAULT_ANNOTATION_LINE_COLOR,
   DEFAULT_ANNOTATION_LINE_WIDTH,
+  DEFAULT_ANTI_ALIASING,
   DEFAULT_BACKGROUND_IMAGE,
   DEFAULT_CAMERA_IS_FIXED,
   DEFAULT_COLOR_ACTIVE,
@@ -70,6 +71,7 @@ import {
   DEFAULT_OPACITY_INACTIVE_MAX,
   DEFAULT_OPACITY_INACTIVE_SCALE,
   DEFAULT_PERFORMANCE_MODE,
+  DEFAULT_PIXEL_ALIGNED,
   DEFAULT_POINT_CONNECTION_COLOR_ACTIVE,
   DEFAULT_POINT_CONNECTION_COLOR_BY,
   DEFAULT_POINT_CONNECTION_COLOR_HOVER,
@@ -234,6 +236,8 @@ const createScatterplot = (
 
   let {
     renderer,
+    antiAliasing = DEFAULT_ANTI_ALIASING,
+    pixelAligned = DEFAULT_PIXEL_ALIGNED,
     backgroundColor = DEFAULT_COLOR_BG,
     backgroundImage = DEFAULT_BACKGROUND_IMAGE,
     canvas = document.createElement('canvas'),
@@ -1464,6 +1468,7 @@ const createScatterplot = (
     );
   };
 
+  const getAntiAliasing = () => antiAliasing;
   const getResolution = () => [canvas.width, canvas.height];
   const getBackgroundImage = () => backgroundImage;
   const getColorTex = () => colorTex;
@@ -1519,6 +1524,7 @@ const createScatterplot = (
   const getIsOpacityByDensity = () => +(opacityBy === 'density');
   const getIsSizedByZ = () => +(sizeBy === 'valueZ');
   const getIsSizedByW = () => +(sizeBy === 'valueW');
+  const getIsPixelAligned = () => +pixelAligned;
   const getColorMultiplicator = () => {
     if (colorBy === 'valueZ') {
       return valueZDataType === CONTINUOUS ? pointColor.length - 1 : 1;
@@ -1632,6 +1638,7 @@ const createScatterplot = (
       },
 
       uniforms: {
+        antiAliasing: getAntiAliasing,
         resolution: getResolution,
         modelViewProjection: getModelViewProjection,
         devicePixelRatio: getDevicePixelRatio,
@@ -1656,11 +1663,14 @@ const createScatterplot = (
         isOpacityByDensity: getIsOpacityByDensity,
         isSizedByZ: getIsSizedByZ,
         isSizedByW: getIsSizedByW,
+        isPixelAligned: getIsPixelAligned,
         colorMultiplicator: getColorMultiplicator,
         opacityMultiplicator: getOpacityMultiplicator,
         opacityDensity: getOpacityDensity,
         sizeMultiplicator: getSizeMultiplicator,
         numColorStates: COLOR_NUM_STATES,
+        drawingBufferWidth: (context) => context.drawingBufferWidth,
+        drawingBufferHeight: (context) => context.drawingBufferHeight,
       },
 
       count: getNumPoints,
@@ -3273,6 +3283,14 @@ const createScatterplot = (
     renderer.gamma = newGamma;
   };
 
+  const setAntiAliasing = (newAntiAliasing) => {
+    antiAliasing = Number(newAntiAliasing) || 0.5;
+  };
+
+  const setPixelAligned = (newPixelAligned) => {
+    pixelAligned = Boolean(newPixelAligned);
+  };
+
   /** @type {<Key extends keyof import('./types').Properties>(property: Key) => import('./types').Properties[Key] } */
   const get = (property) => {
     checkDeprecations({ property: true });
@@ -3608,10 +3626,18 @@ const createScatterplot = (
       return annotationHVLineLimit;
     }
 
+    if (property === 'antiAliasing') {
+      return antiAliasing;
+    }
+
+    if (property === 'pixelAligned') {
+      return pixelAligned;
+    }
+
     return undefined;
   };
 
-  /** @type {(properties: Partial<import('./types').Settable>) => void} */
+  /** @type {(properties: Partial<import('./types').Settable>) => Promise<void>} */
   const set = (properties = {}) => {
     checkDeprecations(properties);
 
@@ -3878,6 +3904,14 @@ const createScatterplot = (
       setAnnotationHVLineLimit(properties.annotationHVLineLimit);
     }
 
+    if (properties.antiAliasing !== undefined) {
+      setAntiAliasing(properties.antiAliasing);
+    }
+
+    if (properties.pixelAligned !== undefined) {
+      setPixelAligned(properties.pixelAligned);
+    }
+
     // setWidth and setHeight can be async when width or height are set to
     // 'auto'. And since draw() would have anyway been async we can just make
     // all calls async.
@@ -4027,9 +4061,94 @@ const createScatterplot = (
     }
   };
 
-  /** @type {() => ImageData} */
-  const exportFn = () =>
-    canvas.getContext('2d').getImageData(0, 0, canvas.width, canvas.height);
+  /**
+   * Export view as `ImageData` using custom render settings
+   * @param {import('./types').ScatterplotMethodOptions['export']} options
+   * @returns {Promise<ImageData>}
+   */
+  const exportFnAdvanced = async (options) => {
+    canvas.style.userSelect = 'none';
+
+    const dpr = window.devicePixelRatio;
+
+    const currPointSize = pointSize;
+    const currWidth = width;
+    const currHeight = height;
+    const currRendererWidth = renderer.canvas.width / dpr;
+    const currRendererHeight = renderer.canvas.height / dpr;
+    const currPixelAligned = pixelAligned;
+    const currAntiAliasing = antiAliasing;
+
+    const scale = options?.scale || 1;
+    const newPointSize = Array.isArray(pointSize)
+      ? pointSize.map((s) => s * scale)
+      : pointSize * scale;
+    const newWidth = currentWidth * scale;
+    const newHeight = currentHeight * scale;
+
+    setPointSize(newPointSize);
+    setWidth(newWidth);
+    setHeight(newHeight);
+    setPixelAligned(options?.pixelAligned || pixelAligned);
+    setAntiAliasing(options?.antiAliasing || antiAliasing);
+
+    renderer.resize(width, height);
+    renderer.refresh();
+
+    await new Promise((resolve) => {
+      pubSub.subscribe('draw', resolve);
+      redraw();
+    });
+
+    const imageData = canvas
+      .getContext('2d')
+      .getImageData(0, 0, canvas.width, canvas.height);
+
+    renderer.resize(currRendererWidth, currRendererHeight);
+    renderer.refresh();
+
+    setPointSize(currPointSize);
+    setWidth(currWidth);
+    setHeight(currHeight);
+    setPixelAligned(currPixelAligned);
+    setAntiAliasing(currAntiAliasing);
+
+    await new Promise((resolve) => {
+      pubSub.subscribe('draw', resolve);
+      redraw();
+    });
+
+    canvas.style.userSelect = null;
+
+    return imageData;
+  };
+
+  /**
+   * Export view as `ImageData` using the current render settings
+   * @overload
+   * @param {undefined} options
+   * @return {ImageData}
+   */
+  /**
+   * Export view as `ImageData` using custom render settings
+   * @overload
+   * @param {import('./types').ScatterplotMethodOptions['export']} options
+   * @return {Promise<ImageData>}
+   */
+  /**
+   * Export view
+   * @param {import('./types').ScatterplotMethodOptions['export']} [options]
+   * @returns {Promise<ImageData>}
+   */
+  const exportFn = (options) => {
+    if (options === undefined) {
+      return canvas
+        .getContext('2d')
+        .getImageData(0, 0, canvas.width, canvas.height);
+    }
+
+    return exportFnAdvanced(options);
+  };
 
   const init = () => {
     updateViewAspectRatio();
diff --git a/src/point.fs b/src/point.fs
index 7870988..4b6cfec 100644
--- a/src/point.fs
+++ b/src/point.fs
@@ -1,6 +1,8 @@
 const FRAGMENT_SHADER = `
 precision highp float;
 
+uniform float antiAliasing;
+
 varying vec4 color;
 varying float finalPointSize;
 
@@ -11,7 +13,7 @@ float linearstep(float edge0, float edge1, float x) {
 void main() {
   vec2 c = gl_PointCoord * 2.0 - 1.0;
   float sdf = length(c) * finalPointSize;
-  float alpha = linearstep(finalPointSize + 0.5, finalPointSize - 0.5, sdf);
+  float alpha = linearstep(finalPointSize + antiAliasing, finalPointSize - antiAliasing, sdf);
 
   gl_FragColor = vec4(color.rgb, alpha * color.a);
 }
diff --git a/src/point.vs b/src/point.vs
index 3f07999..c4cb35c 100644
--- a/src/point.vs
+++ b/src/point.vs
@@ -23,12 +23,15 @@ uniform float isOpacityByW;
 uniform float isOpacityByDensity;
 uniform float isSizedByZ;
 uniform float isSizedByW;
+uniform float isPixelAligned;
 uniform float colorMultiplicator;
 uniform float opacityMultiplicator;
 uniform float opacityDensity;
 uniform float sizeMultiplicator;
 uniform float numColorStates;
 uniform float pointScale;
+uniform float drawingBufferWidth;
+uniform float drawingBufferHeight;
 uniform mat4 modelViewProjection;
 
 attribute vec2 stateIndex;
@@ -39,7 +42,17 @@ varying float finalPointSize;
 void main() {
   vec4 state = texture2D(stateTex, stateIndex);
 
-  gl_Position = modelViewProjection * vec4(state.x, state.y, 0.0, 1.0);
+  if (isPixelAligned < 0.5) {
+    gl_Position = modelViewProjection * vec4(state.x, state.y, 0.0, 1.0);
+  } else {
+    vec4 clipSpacePosition = modelViewProjection * vec4(state.x, state.y, 0.0, 1.0);
+    vec2 ndcPosition = clipSpacePosition.xy / clipSpacePosition.w;
+    vec2 pixelPos = 0.5 * (ndcPosition + 1.0) * vec2(drawingBufferWidth, drawingBufferHeight);
+    pixelPos = floor(pixelPos + 0.5); // Snap to nearest pixel
+    vec2 snappedPosition = (pixelPos / vec2(drawingBufferWidth, drawingBufferHeight)) * 2.0 - 1.0;
+    gl_Position = vec4(snappedPosition, 0.0, 1.0);
+  }
+
 
   // Determine color index
   float colorIndexZ =  isColoredByZ * floor(state.z * colorMultiplicator);
diff --git a/src/renderer.js b/src/renderer.js
index 09025f1..fdc1186 100644
--- a/src/renderer.js
+++ b/src/renderer.js
@@ -144,7 +144,10 @@ export const createRenderer = (
     }
   });
 
-  const resize = () => {
+  const resize = (
+    /** @type {number} */ customWidth,
+    /** @type {number} */ customHeight,
+  ) => {
     // We need to limit the width and height by the screen size to prevent
     // a bug in VSCode where the window height is said to be taller than the
     // screen height. The problem with too large dimensions is that at some
@@ -157,8 +160,14 @@ export const createRenderer = (
     // @see
     // https://github.com/microsoft/vscode/issues/225808
     // https://github.com/flekschas/jupyter-scatter/issues/37
-    const width = Math.min(window.innerWidth, window.screen.availWidth);
-    const height = Math.min(window.innerHeight, window.screen.availHeight);
+    const width =
+      customWidth === undefined
+        ? Math.min(window.innerWidth, window.screen.availWidth)
+        : customWidth;
+    const height =
+      customHeight === undefined
+        ? Math.min(window.innerHeight, window.screen.availHeight)
+        : customHeight;
     canvas.width = width * window.devicePixelRatio;
     canvas.height = height * window.devicePixelRatio;
     fboRes[0] = canvas.width;
@@ -166,9 +175,13 @@ export const createRenderer = (
     fbo.resize(...fboRes);
   };
 
+  const resizeHandler = () => {
+    resize();
+  };
+
   if (!options.canvas) {
-    window.addEventListener('resize', resize);
-    window.addEventListener('orientationchange', resize);
+    window.addEventListener('resize', resizeHandler);
+    window.addEventListener('orientationchange', resizeHandler);
     resize();
   }
 
@@ -177,8 +190,8 @@ export const createRenderer = (
    */
   const destroy = () => {
     isDestroyed = true;
-    window.removeEventListener('resize', resize);
-    window.removeEventListener('orientationchange', resize);
+    window.removeEventListener('resize', resizeHandler);
+    window.removeEventListener('orientationchange', resizeHandler);
     frame.cancel();
     canvas = undefined;
     regl.destroy();
@@ -229,6 +242,7 @@ export const createRenderer = (
       return isDestroyed;
     },
     render,
+    resize,
     onFrame,
     refresh,
     destroy,
diff --git a/src/types.d.ts b/src/types.d.ts
index 415b99d..59b7fbf 100644
--- a/src/types.d.ts
+++ b/src/types.d.ts
@@ -165,6 +165,8 @@ interface BaseOptions {
   yScale: null | Scale;
   pointScaleMode: PointScaleMode;
   cameraIsFixed: boolean;
+  antiAliasing: number;
+  pixelAligned: boolean;
 }
 
 // biome-ignore lint/style/useNamingConvention: KDBush is a library name
@@ -265,6 +267,11 @@ export interface ScatterplotMethodOptions {
     transitionDuration: number;
     transitionEasing: (t: number) => number;
   }>;
+  export: Partial<{
+    scale: number;
+    antiAliasing: number;
+    pixelAligned: boolean;
+  }>;
 }
 
 export type Events = import('pub-sub-es').Event<


Test patch:
diff --git a/tests/methods.test.js b/tests/methods.test.js
index 14fe657..b3e020f 100644
--- a/tests/methods.test.js
+++ b/tests/methods.test.js
@@ -896,3 +896,35 @@ test('pointScaleMode', async () => {
 
   scatterplot.destroy();
 });
+
+test('export()', async () => {
+  const dim = 10;
+  const scatterplot = createScatterplot({
+    canvas: createCanvas(dim, dim),
+    width: dim,
+    height: dim,
+    pointSize: 4,
+  });
+
+  await scatterplot.draw([[0.01, 0.01]]);
+
+  const initialImage = scatterplot.export();
+  const initialPixelSum = getPixelSum(initialImage, 0, dim, 0, dim);
+
+  expect(initialPixelSum).toBeGreaterThan(0);
+
+  // Export at two scale
+  const upscaledImage = await scatterplot.export({ scale: 2 });
+  const upscaledPixelSum = getPixelSum(upscaledImage, 0, dim * 2, 0, dim * 2);
+  expect(upscaledPixelSum).toBeGreaterThan(initialPixelSum);
+
+  // Align point with pixel grid
+  const pixelAlignedImage = await scatterplot.export({ pixelAligned: true });
+  expect(pixelAlignedImage.data).not.toEqual(initialImage.data);
+
+  // Increase anti aliasing
+  const antiAliasingImage = await scatterplot.export({ antiAliasing: 2 });
+  const antiAliasingPixelSum = getPixelSum(antiAliasingImage, 0, dim, 0, dim);
+
+  expect(antiAliasingPixelSum).toBeLessThan(initialPixelSum);
+});