Repository: vispy/vispy

Issue:
Markers scaling behavior broken
Hello everybody, I'd say, this breaks documented functionality, namely [MarkersVisual.scaling](https://vispy.org/api/vispy.visuals.markers.html#vispy.visuals.markers.MarkersVisual.scaling) e.g. in [examples/basics/visuals/markers.py](https://github.com/vispy/vispy/blob/c0d7457a7b1717d47baa8b8622faa2c67bbe4be0/examples/basics/visuals/markers.py#L63): Pressing 's' should toggle between the markers remaining the same size when zooming and scaling with the zoom.

This works with version 0.11.0, but not with version 0.12.1

_Originally posted by @arcanerr in https://github.com/vispy/vispy/issues/2359#issuecomment-1433559707_
            


Discussion:
Yeah this is bad. Probably explains some things @ameraner was seeing with one of our applications (SIFT) related to the newer versions of SIFT.
Ok so let's lay out the behaviors and expectations. As documented in the MarkersVisual, the `scaling` parameter means (I have to look this up every time this comes up):

* False (default): The markers will be the same number of pixels regardless of zoom level.
* True: The markers will shrink or grow as you zoom as if you were moving them closer to/further away from the screen.

The problem with the previous implementation was that it used the MarkersVisuals transform to determine this so in 3D views you would see markers change in size just by rotating the view (I think that's what we had figured out).

The change implemented in #2359 had the side effect that MarkersVisuals `.transform` no longer changed the *size* of the Markers, but only the vertex coordinates (the location of the Markers).

I see now that the referenced example is doing exactly that, using the Markers transform to change the "zoom" of the Visual:

https://github.com/vispy/vispy/blob/c0d7457a7b1717d47baa8b8622faa2c67bbe4be0/examples/basics/visuals/markers.py#L47-L49
So the question is, how do we update the example to reflect the expected behavior? In a SceneCanvas setting you would maybe change a higher-up transform (a camera) and expect to see the scaling change (not sure that still works). Or do we change/revert some/all of the changes of #2359 to have the Visual transform have some effect on size?
The SceneCanvas-based version of the same example works:

```python
import numpy as np

from vispy import app
from vispy.scene import SceneCanvas, visuals
from vispy.visuals.transforms import STTransform

n = 500
pos = np.zeros((n, 2))
colors = np.ones((n, 4), dtype=np.float32)
radius, theta, dtheta = 1.0, 0.0, 5.5 / 180.0 * np.pi
for i in range(500):
    theta += dtheta
    x = 256 + radius * np.cos(theta)
    y = 256 + radius * np.sin(theta)
    r = 10.1 - i * 0.02
    radius -= 0.45
    pos[i] = x, y
    colors[i] = (i/500, 1.0-i/500, 0, 1)


class Canvas(SceneCanvas):
    def __init__(self):
        super().__init__(keys='interactive', size=(512, 512),
                title="Marker demo [press space to change marker]")
        self.unfreeze()
        self.index = 0
        self.view = self.central_widget.add_view()
        self.view.camera = 'panzoom'
        self.view.camera.set_range(x=(0, 500), y=(0, 500))
        self.markers = visuals.Markers(parent=self.view.scene)
        self.markers.set_data(pos, face_color=colors)
        self.markers.symbol = self.markers.symbols[self.index]
        #self.markers.transform = STTransform()
        self.freeze()

        self.show()

    def on_key_press(self, event):
        if event.text == ' ':
            self.index = (self.index + 1) % (len(self.markers.symbols))
            self.markers.symbol = self.markers.symbols[self.index]
            self.update()
        elif event.text == 's':
            self.markers.scaling = not self.markers.scaling
            print(self.markers.scaling)
            self.update()


if __name__ == '__main__':
    canvas = Canvas()
    app.run()
```

Basically the transform logic happens on the cameras transform, not directly on the MarkersVisual.
Before #2359, changing the `scale` of a point affected the marker `size`. This is sometimes desirable in 2D (such as in the example above), and sometimes it is not (such as if coordinates need to be rescaled to match an image with a specific pixel size, but the user wants to give marker sizes with the destination size, rather than the original).

However, most the most important complication comes from also supporting anisotropic scales: if the scale components of a transform are different in x and y, then the previous system is simply wrong: what does it mean to rescale the size of a `disc` by `(5, 2)`? Should it become an ellipse? Since this is not really possible with GL Markers (unless we go *much* more complicated on the shader), the only sensible solution is to *not* have the marker size be affected by scaling. (This is more obvious with 3D data, where rotating the camera affects the vectors used for size calculations in the shader, but it affects any anisotropic data).

Basically:

```py
coords = [
	[ 0,  0],
	[ 0, 10],
]
```

this with a size of `10`, makes two "touching" circles. If you the provide a scale of `(2, 3)`, what do you expect to get out?

---

As you pointed out, the reason why the example above works in `SceneCanvas` version is because it's the camera that changes, and not the visual. I think that the "simpler" example without canvas is just confusing at this point, and should be replaced by the one you mentioned above.

PS: if you want to get the old behaviour back, you can just multiply the `size` of the marker by the scale and you're good to go :)
> you can just multiply the size of the marker by the scale and you're good to go :)

@brisvag you mean the scale of the transform being used? And that would have to be done for each update, right?
Correct! Basically add:

```py
    def on_mouse_wheel(self, event):
        """Use the mouse wheel to zoom."""
        zoom_amount = 1.25**event.delta[1]
        self.markers.transform.zoom((zoom_amount,) * 2, center=event.pos)
        self._marker_size *= zoom_amount
        self.markers.set_data(pos, face_color=colors, size=self._marker_size)
        self.update()
```

Or something like this. Note however that you would actually need some extra logic if you want to swap between scaling and non-scaling (disable the size change when scaling is False basically).
However, I feel like changing the example like this is making it harder and harder to follow...
@brisvag The issue we're running into in our application is that we have a PanZoom camera which controls the general pan/zoom nature of the SceneCanvas, but we have a series of Nodes with Visuals as children. These Nodes have map projection transforms on them and they can change between different geographic projections (ex. mercator versus polar-stereographic versus other). So the pixel size in one projection does not work in another. I suppose our short term fix could be recalculating the pixel size when we switch to a different projection (I think this logically makes sense), but this feels like a major loss in functionality...right? Or maybe it is unrealistic to assume this should work this way. Put another way, why should we (the SIFT project) assume that a screen-based pixel size be effected by coordinate transformations. @arcanerr thoughts?
In general, I think this is really "reducing out assumptions" rather than a loss of functionality. But in your use case (since data is isotropic) it adds an extra step.

I would vote against reverting the change, because while it's making one use case (albeit a commom one) more convenient, it simply breaks another (anisotropic) with especially awful results in 3D. This breakage cannot be undone as easily as adding a callback on zoom. Indeed, if we were to revert here, I think I would end up redoing these changes in napari and maintain a different shader there, as anisotropic data is relatovely common in 3D imaging.
> as anisotropic data is relatovely common in 3D imaging.

Yeah this was going to be my next question. If this is common in other scientific fields then it makes sense to require one use case to have an extra step rather than completely break it for another use case.
> Put another way, why should we (the SIFT project) assume that a screen-based pixel size be effected by coordinate transformations. @arcanerr thoughts?

Basically, I think this questions needs to be asked side by side with:

Why should project X, which uses anisotropic scaling, assume that scaling will affect marker size only on one (arbitrary) axis (or even worse, a linear combination of axes if rotations are applied)?
I think I agree with you @brisvag although I don't want to. For my own understanding, could you explain (simply) one of the use cases that is common in napari that saw the prior issues (anisotropic)? I understand that these cases exist, but don't have an understanding of when or why it would be done (coordinates versus marker size).
I don't personally have such data, but maybe @Czaki can chip in with some real data? (he opened the original issue at https://github.com/napari/napari/issues/2213, where you can see the effect in action in 3D).

Note that while the "zoomin in and out" issue will only be obvious in 3D (cause of rotations), this is also an issue if you have 2D anisotropic data; it might even be worse, since it won't be "shifting" and you might not notice that (or understand why) the size of your points is incorrect.
In my case, it happens for any data with a scale 3,1,1 (210,70,70). When working with data of scale 5,1,1 (300, 50, 50), it was even more visible. But as I remember, people on macOS do not meet this problem. Maybe there are some differences in handling this scale between OS.  
Ok, I'll try to put my two cents in by thinking out loud about what I would expect `.scaling` to be interpreted as when `Markers` are **used in a scene graph** (in contrast to using the Visuals directly, I don't have experience with that setup):

1. `scaling=False` should help to implement kind of a special case, that is: Markers are _positioned_ with respect to the accumulated transformations of the scene and the camera, but their shapes are rendered directly in screen space (that is equivalent to that the linear 3x3 part of the accumulated homogeneous transformation matrix[^1] is ignored). Side note: all fragments have to take the depth of the reference point i.e. their position to update the z-Buffer. I guess that's how it is in the current implementation.
2. With `scaling=True` - uhm - the only consistent interpretation I see for that in a _3D_ scene graph is: if there is anisotropic scaling or even non-linear distortion in the accumulated transformation, Markers should be scaled anisotropically or distorted :thinking:. If you want to implement a different interpretation, you have to ask: Should it be interpreted as `scaling==True and rotation==False and shearing==False and reflection==False ...`?!? It doesn't seem to make sense to ignore only parts of the linear matrix in the affine case, for a nonlinear transformation like map projections it makes even less sense, does it? Is this the same interpretation you prefer, @brisvag?

I'd say, with this interpretation the SIFT use case cannot be fulfilled without extra work: Here images are displayed as a 2D scene with images as layers in a layer stack (like Photoshop layers), but that is "faked" by setting up a 3D scene with orthographic projection and all the images always facing to the camera. Here we want to have Markers which
1. either stay the same size independent of the zoom (that works with `scaling=False`)
2. or resize when zooming to create the effect, that they keep the same size relative to the displayed images/the scene, but with constant width of the outline (this happens with `scaling=False` with pre-0.12.0 VisPy)

(2) :thinking: could be solved by applying the zoom factor to the Markers coherently with the camera zoom. Is that how it was until the merge request in question?

Unfortunately I won't have the capacity to contribute to this discussion much more.

----
[^1]: From now on, we I briefly refer to this part as the _linear matrix_ 
> With `scaling=True` - uhm - the only consistent interpretation I see for that in a _3D_ scene graph is: if there is anisotropic scaling or even non-linear distortion in the accumulated transformation, Markers should be scaled anisotropically or distorted :thinking:.

Yes, I agree, this is the best scenario. That's what I meant here:

> However, most the most important complication comes from also supporting anisotropic scales: if the scale components of a transform are different in x and y, then the previous system is simply wrong: what does it mean to rescale the size of a `disc` by `(5, 2)`? Should it become an ellipse? Since this is not really possible with GL Markers (unless we go _much_ more complicated on the shader), the only sensible solution is to _not_ have the marker size be affected by scaling. (This is more obvious with 3D data, where rotating the camera affects the vectors used for size calculations in the shader, but it affects any anisotropic data).

As I mentioned here, I fear that this will require significant work on the shader. Maybe I'm wrong, but my first reaction was that my PR was the only solution I could reasonably tackle :P And this is only accounting 2D; if you try to do this in 3D, you effectively have to change discs to ellipsoids and project them according to rotation. Which *maybe* is possible? But certainly hard and too much work for me to justify taking on.

> If you want to implement a different interpretation, you have to ask: Should it be interpreted as `scaling==True and rotation==False and shearing==False and reflection==False ...`?!? It doesn't seem to make sense to ignore only parts of the linear matrix in the affine case, for a nonlinear transformation like map projections it makes even less sense, does it? Is this the same interpretation you prefer, @brisvag?

EDIT: see below comment for better breakdown.
~I'm not sure I follow exactly your reasoning here, so forgive me if I misunderstand. I think that what we had before `v0.12` was effectively:~
~- marker coordinates are affected by all components in the matrix~
~- marker shapes are affected only by scaling (no rotation, no reflection, but shear is part of scale right? Sorry, my math theory is weak here :P). Specifically, the amount of scaling was the projection of the `scale` vector onto the buffer x coordinate, which was then applied to all axes (assuming isotropicity). This is what causes the weird behaviour on rotation in 3D (and would also do the same if rotating in 2D, it's just that we typically don't do that).~

After my PR, the marker shapes are instead fully unaffected by the matrix. I think the only better solution is to have them *fully* affected by the matrix.

As a general thought: we need to keep in mind that markers are a *screen space* visual, much like little banners that follow the camera. In that sense, I feel like we're trying to force them into something they're not. If we decide to go for the "full matrix" approach, we should likely abandon gl markers and do everything ourselves (which could also solve #2078).
Rereading what I wrote, I'm thinking that maybe I got it wrong. I'll try again :P

I think some confusion comes from the fact that we have multiple transforms at play. We don't simply "use" or "ignore" entirely a part of *the* matrix. Instead, we use/ignore some parts of some of the matrices that compose the full matrix.

Before `v0.12`:

- marker coordinates are affected by the full transform from `visual` to `framebuffer`.
- marker shapes are also affected by the transform from `visual` to `framebuffer`, but the `scale` and `shear` components are used in a weird way (the amount of scaling was the projection of the scale vector onto the buffer x coordinate, which was then applied to all axes (assuming isotropicity). This is what causes the weird behaviour on rotation in 3D (and would also do the same if rotating in 2D, it's just that we typically don't do that). While this seems weird, this is (one of the?) only sensible way to rescale markers that are *isotropic and fixed in shape*. This would not be a problem if we can get markers that properly shear and project in 3D (as mentioned above: *hard*).

After `v0.12`:
- marker coordinates are affected by the full transform from `visual` to `framebuffer`.
- marker shapes are only affected by the transform from `scene` to `framebuffer`. This means that any shear that happens on the `Visual` level is ignored, as well as any scaling. However, other transforms (such as the camera zoom) are used properly. In hindsight, this also probably means that if somehow another shearing transform is introduced between `scene` and `framebuffer`, the same issue as before comes up :/.

> (2) thinking could be solved by applying the zoom factor to the Markers coherently with the camera zoom. Is that how it was until the merge request in question?

Yes, this is how you can solve your problem by just adding a callback that sets the size based on zoom. But this [*should* still work in your use case](https://github.com/vispy/vispy/issues/2453#issuecomment-1433613812), since you're using the `SceneCanvas` and not just the `Visuals`. The only difference after `v0.12` is that you have to change the size of the markers if you change their transform.
> - marker shapes are only affected by the transform from scene to framebuffer. This means that any shear that happens on the Visual level is ignored, as well as any scaling. However, other transforms (such as the camera zoom) are used properly. In hindsight, this also probably means that if somehow another shearing transform is introduced between scene and framebuffer, the same issue as before comes up :/.

What I don't understand then what's the difference between `.scaling=True` and `.scaling=False` in v0.12? There is one, but I was not able to comprehend the implemented logic from the results.
>  This would not be a problem if we can get markers that properly shear and project in 3D (as mentioned above: hard).

One could think about to place polygons into the scene and render the markers similar to textures onto them, then the polygons could be transformed like all the other primitives. This won't work that well with the `sphere` Markers though. But yes, without knowing the internals of VisPy, this means a completely different setup than the current one and therefore can be called _hard_.
I'll be honest, I'm quickly losing track of an understanding of this discussion. But I think I still have questions/comments that are appropriate:

> marker shapes are only affected by the transform from scene to framebuffer. This means that any shear that happens on the Visual level is ignored, as well as any scaling. However, other transforms (such as the camera zoom) are used properly. In hindsight, this also probably means that if somehow another shearing transform is introduced between scene and framebuffer, the same issue as before comes up :/.

How do (or should) Markers behave for non-linear cameras? We have strange cameras like the [magnify camera](https://github.com/vispy/vispy/blob/main/vispy/scene/cameras/magnify.py) that although it is more of a gimmick/special case, it does produce some interesting visualizations and use case of a non-linear camera transformation.

Which leads to another point that is maybe being implicitly said here: Markers represent single points in space. Meaning the shape of the Marker (the symbol) ~does~doesn't actually exist, it is just an identifier/marker/indicator of a point in space. But is this true (do users expect it to be ture) with scaling=False and scaling=True?

> What I don't understand then what's the difference between .scaling=True and .scaling=False in v0.12? There is one, but I was not able to comprehend the implemented logic from the results.

After the various discussions on the recent changes I would expect this could be summarized as `scaling=False` should result in Markers (again, points in space) whose location and size is determined by all transforms involved; the "scene" gets smaller, the markers get smaller. So `scaling=True` is scene gets smaller, the markers stay the same size, but same location. I think by this simple definition this means that `scaling=False` should have Markers whose shapes change with all transforms and possibly skewed into weird looking shapes while `scaling=True` should always be the same basic shape (isotropic? a circle is always a circle on the 2D screen). I think with the current implementation this last description will never be true for all cameras, right? The cameras are still "skewing" the "perfect" shape of the Marker.

Sorry if I'm completely missing the point here. I'm still not sure I have all the use cases properly represented in my head. I wasn't looking for actual data in my previous comment, more just "I look at data of X which represent Y so they have to scale differently in one dimension to be as close to real world as possible".
Side question: @arcanerr with the current implementation, could SIFT set the size of the Marker whenever the viewing projection is changed? Or because of possibly non-linear map projections that size would have to be determined for every location of the Marker (a marker on the equator of a mercator projection is one size but that same size marker on the high latitudes of the mercator projection would be much larger)?
> What I don't understand then what's the difference between `.scaling=True` and `.scaling=False` in v0.12? There is one, but I was not able to comprehend the implemented logic from the results.

In short, the difference is that the `scale` component of the transform in `Markers.transform` is ignored when calculating marker size, while `scale` components of other transforms (i.e: `camera`) are still used. This was under the (probably wrong) assumption that anisotropic transforms are introduced at the `Visual` level.

Ultimately, it was wrong before, and it is still wrong; I just find the "new wrong" easier to workaround (with an event callback only on transform changes) than the old one (which would require much more complex event that fires even on camera rotation, for example). (Note that it's not just me "not needing" the old behaviour; in fact I already am using this exact workaround somehwere in my code).

---

Non-linear cameras should still still work the same (i.e: the same as before, with the anisotropic issue included), as well as non-linear transform *if* they are applied to the whole scene, rather than just one visual.

---

> Which leads to another point that is maybe being implicitly said here: Markers represent single points in space. Meaning the shape of the Marker (the symbol) does actually exist, it is just an identifier/marker/indicator of a point in space. But is this true (do users expect it to be ture) with scaling=False and scaling=True?

Exactly the source of the problem: as soon as you use markers to represent *physical* size, that's where all the issues begin... but we don't have anything better at the moment :P. That's where we should probably have a separate visual that handles any projection even in 3D anisotropic.

I agree that this all feels very arbitrary (and is ultimately wrong), but realistically I can't afford to spend time any time soon rewriting the whole `Markers` visual to do the fancy thing :( I think it's realtively doable in 2D, but in 3D you need a full ellipsoid that can rotate in 3 dimensions...
> Side question: @arcanerr with the current implementation, could SIFT set the size of the Marker whenever the viewing projection is changed? Or because of possibly non-linear map projections that size would have to be determined for every location of the Marker (a marker on the equator of a mercator projection is one size but that same size marker on the high latitudes of the mercator projection would be much larger)?

I guess it should be possible: there must be code which is triggered when zoom interaction happens (and a different Projection is selected!), at this/these point(s) I'd try to update `Markers.size`. But since I did not inspect the PanZoom camera code I can't tell exactly how to calculate the correct value.

A problem is also, that when you do this repeatedly (something like `my_markers.size = calc_new_size(my_markers.size)`) numerical errors can accumulate, i.e., the size of the Markers may drift in a way that even though the scene may appear in the same size on the screen at two points in time the markers could be smaller or larger than initially. To avoid this, the relative (to the scene) marker size needs to be stored and the correct way to update the size would be something like `my_markers.size = calc_size( my_markers.relative_size)` :thinking: ...

An alternative could be to reactivate the old code as a 3rd code path parallel to the new `scaling=True` code path (as you already state [here](https://github.com/vispy/vispy/pull/2359#discussion_r932380294), @brisvag). To tell the `Markers` when to use this path, a new property (`legacy_scaling=True`, `screen_space_scaling=True` or ...?) would be required. Or you introduce such a property for the new code path and revive the old code path for `scaling=True`...


> An alternative could be to reactivate the old code as a 3rd code path parallel to the new `scaling=True` code path (as you already state [here](https://github.com/vispy/vispy/pull/2359#discussion_r932380294), @brisvag).

Yes, that's definitely an option. I don't love the idea to have not one but two code paths which are both wrong, but if we can't come up with anything better that's better than nothing :P
Note: I realized in my last big comment I said "does" and meant "doesn't". Looks like @brisvag understood what I was going for. The shape/symbol of the marker doesn't actually exist. It is just that, a symbol for a single point. I don't think the Markers in our SIFT application are treating them this way (right @arcanerr?), just that users are or will be surprised when Marker size changes between views when they are meant to represent the same thing in the same exact way but the user is only requesting to see them from a different "angle" (projection).

> Yes, that's definitely an option. I don't love the idea to have not one but two code paths which are both wrong, but if we can't come up with anything better that's better than nothing :P

Maybe not two wrong approaches, but approaches with more specific descriptions or identifiers/flags. I mean, could we identify this as different sizing modes/methods for the Markers? I think no matter how this issue is moved forward the description of the Markers needs to be updated to clearly state that Markers are points in space, not something that exists with that particular shape/size in the "world" being viewed...Although now that I say that we have this new-ish example which uses sphere Markers for this exact purpose:

https://vispy.org/gallery/scene/marker_spheres.html#sphx-glr-gallery-scene-marker-spheres-py

So maybe this description can't be something we depend on (marker is a point, not a shape).

Anyway...we already have a `scaling` keyword argument which in English could be used to describe something with more than a boolean False/True set of options. I'm not sure what the shader looks like in this sense, but if it makes it easier we could require re-compiling the shader by swapping out scaling functions: no-op for scaling==False, old way of scaling for ==True, the new way of scaling for scaling=='some_string_name', or something like that. Basically, False and True are there for backwards compatibility but new string modes are there for controlling what transforms have an effect on the shape of the Markers?
> Anyway...we already have a `scaling` keyword argument which in English could be used to describe something with more than a boolean False/True set of options. I'm not sure what the shader looks like in this sense, but if it makes it easier we could require re-compiling the shader by swapping out scaling functions: no-op for scaling==False, old way of scaling for ==True, the new way of scaling for scaling=='some_string_name', or something like that. Basically, False and True are there for backwards compatibility but new string modes are there for controlling what transforms have an effect on the shape of the Markers?

Sounds good to me. I'm not yet happy with the following suggestion for the names of the scaling modes, but as a starting point, how about these?:
- `"fixed"` or `"off"`:  equivalent to `scaling=False`
- `"scene"`: equivalent to `scaling=True` in pre-v0.12
- `"camera"`: equivalent to `scaling=True` in v0.12+

And the next question: should `scaling=True` revert to its pre-v0.12 meaning or keep the new v0.12+ one? 
I vote for the revert because it is the longer existing interpretation, so there is probably more code in the wild implemented for it (and it would fix a SIFT bug without further ado :wink:).
What about instead of `scene` using `visual`? Isn't that more accurate in the sense that it follows the transforms affecting the Visual object?

I could see "camera" being confusing to people who aren't directly using a camera, but I'm also hoping there is a more descriptive term that reflects the "non-scene graph transforms".

> And the next question: should scaling=True revert to its pre-v0.12 meaning or keep the new v0.12+ one?
I vote for the revert because it is the longer existing interpretation

I agree for the same reasons, but also because the first group to identify this problem (napari) are the most active and most likely to customize vispy as needed in their own distribution.

@brisvag Thoughts? Alternatives (naming of the above scaling options or entirely different implementations)?
To be honest, the more I think about this and the more I hate my "fix" from #2359 xD But in the interest of having *something* that works, I guess the above proposal is ok for me.

In that case, I would go for `False` and `True` falling back to previous behaviour, and `fixed`/`visual`/`scene` for the string options.
>  the more I hate my "fix"

What does your perfect solution that you don't hate look like? Does it involve some complex shader code or does it look like a lot of user controllable options in the MarkersVisual or something else?
To be clear, I'm not trying to block anything here, I'm just annoyed that what I thought was a good fix turned out to be just as arbitrary as the previous behaviour ^^'

In an ideal world, I think the elegant solution would be to simply *not* have the `scaling` option on markers, and have them just be "indicator of a zero-dimensional point in space", while handling physically "sized" objects with instanced rendering of simple meshes (or SDF impostors like spheres) which would allow us to use the *whole* matrix.
Maybe a way forward in #2460, of we'ok with giving up *some* things in exchange for correctly transformed "points".

Resolving patch:
diff --git a/examples/basics/visuals/markers.py b/examples/basics/visuals/markers.py
--- a/examples/basics/visuals/markers.py
+++ b/examples/basics/visuals/markers.py
@@ -4,7 +4,13 @@
 # Copyright (c) Vispy Development Team. All Rights Reserved.
 # Distributed under the (new) BSD License. See LICENSE.txt for more info.
 # -----------------------------------------------------------------------------
-""" Display markers at different sizes and line thicknessess.
+"""Display markers at different sizes and line thicknesses.
+
+Keyboard options:
+* spacebar: Cycle through possible marker symbols.
+* "s": Switch between "fixed" marker scaling (initial setting) and "scene"
+  scaling.
+
 """
 
 import numpy as np
@@ -60,10 +66,11 @@ def on_key_press(self, event):
             self.markers.symbol = self.markers.symbols[self.index]
             self.update()
         elif event.text == 's':
-            self.markers.scaling = not self.markers.scaling
+            self.markers.scaling = "fixed" if self.markers.scaling != "fixed" else "scene"
             self.update()
 
 
 if __name__ == '__main__':
+    print(__doc__)
     canvas = Canvas()
     app.run()
diff --git a/vispy/visuals/markers.py b/vispy/visuals/markers.py
--- a/vispy/visuals/markers.py
+++ b/vispy/visuals/markers.py
@@ -16,7 +16,7 @@
 _VERTEX_SHADER = """
 uniform float u_antialias;
 uniform float u_px_scale;
-uniform bool u_scaling;
+uniform int u_scaling;
 uniform bool u_spherical;
 
 attribute vec3 a_position;
@@ -43,28 +43,38 @@
 
     vec4 pos = vec4(a_position, 1);
     vec4 fb_pos = $visual_to_framebuffer(pos);
+    vec4 x;
+    vec4 size_vec;
     gl_Position = $framebuffer_to_render(fb_pos);
 
     // NOTE: gl_stuff uses framebuffer coords!
-
-    if (u_scaling == true) {
-        // calculate point size from visual to framebuffer coords to determine size
+    if (u_scaling == 1) {
+        // scaling == "scene": scale marker using entire visual -> framebuffer set of transforms
+        x = $framebuffer_to_visual(fb_pos + vec4(big_float, 0, 0, 0));
+        x = (x - pos);
+        size_vec = $visual_to_framebuffer(pos + normalize(x) * a_size);
+        $v_size = size_vec.x / size_vec.w - fb_pos.x / fb_pos.w;
+        v_edgewidth = ($v_size / a_size) * a_edgewidth;
+    }
+    else if (u_scaling == 2) {
+        // scaling == "visual": scale marker using only the Visual's transform
         // move horizontally in framebuffer space
         // then go to scene coordinates (not visual, so scaling is accounted for)
-        vec4 x = $framebuffer_to_scene(fb_pos + vec4(big_float, 0, 0, 0));
+        x = $framebuffer_to_scene(fb_pos + vec4(big_float, 0, 0, 0));
         // subtract position, so we get the scene-coordinate vector describing
         // an "horizontal direction parallel to the screen"
         vec4 scene_pos = $framebuffer_to_scene(fb_pos);
         x = (x - scene_pos);
         // multiply that direction by the size (in scene space) and add it to the position
         // this gives us the position of the edge of the point, which we convert in screen space
-        vec4 size_vec = $scene_to_framebuffer(scene_pos + normalize(x) * a_size);
+        size_vec = $scene_to_framebuffer(scene_pos + normalize(x) * a_size);
         // divide by `w` for perspective, and subtract pos
         // this gives us the actual screen-space size of the point
         $v_size = size_vec.x / size_vec.w - fb_pos.x / fb_pos.w;
         v_edgewidth = ($v_size / a_size) * a_edgewidth;
     }
     else {
+        // scaling == "fixed": marker is always the same number of pixels
         $v_size = a_size * u_px_scale;
         v_edgewidth = a_edgewidth * u_px_scale;
     }
@@ -505,8 +515,20 @@ class MarkersVisual(Visual):
         The color used to draw each symbol interior.
     symbol : str or array
         The style of symbol used to draw each marker (see Notes).
-    scaling : bool
-        If set to True, marker scales when rezooming.
+    scaling : str | bool
+        Scaling method of individual markers. If set to "fixed" (default) then
+        no scaling is done and markers will always be the same number of
+        pixels on the screen. If set to "scene" then the chain of transforms
+        from the Visual's transform to the transform mapping to the OpenGL
+        framebuffer are used to scaling the marker. This has the effect of the
+        marker staying the same size in the "scene" coordinate space and
+        changing size as the visualization is zoomed in and out. If set to
+        "visual" the marker is scaled only using the transform of the Visual
+        and not the rest of the scene/camera. This means that something like
+        a camera changing the view will not affect the size of the marker, but
+        the user can still scale it using the Visual's transform. For
+        backwards compatibility this can be set to the boolean ``False`` for
+        "fixed" or ``True`` for "scene".
     alpha : float
         The opacity level of the visual.
     antialias : float
@@ -534,7 +556,7 @@ class MarkersVisual(Visual):
     _symbol_shader_values = symbol_shader_values
     _symbol_shader = symbol_func
 
-    def __init__(self, scaling=False, alpha=1, antialias=1, spherical=False,
+    def __init__(self, scaling="fixed", alpha=1, antialias=1, spherical=False,
                  light_color='white', light_position=(1, -1, 1), light_ambient=0.3, **kwargs):
         self._vbo = VertexBuffer()
         self._data = None
@@ -554,6 +576,8 @@ def __init__(self, scaling=False, alpha=1, antialias=1, spherical=False,
         if len(kwargs) > 0:
             self.set_data(**kwargs)
 
+        self._scaling = "fixed"
+        self._scaling_int = 0
         self.scaling = scaling
         self.antialias = antialias
         self.light_color = light_color
@@ -677,9 +701,20 @@ def scaling(self):
 
     @scaling.setter
     def scaling(self, value):
-        value = bool(value)
-        self.shared_program['u_scaling'] = value
+        scaling_modes = {
+            False: 0,
+            True: 1,
+            "fixed": 0,
+            "scene": 1,
+            "visual": 2,
+        }
+        if value not in scaling_modes:
+            possible_options = ", ".join(repr(opt) for opt in scaling_modes)
+            raise ValueError(f"Unknown scaling option {value!r}, expected one of: {possible_options}")
+        scaling_int = scaling_modes[value]
+        self.shared_program['u_scaling'] = scaling_int
         self._scaling = value
+        self._scaling_int = scaling_int
         self.update()
 
     @property
@@ -773,7 +808,7 @@ def _prepare_draw(self, view):
         if self._data is None:
             return False
         view.view_program['u_px_scale'] = view.transforms.pixel_scale
-        view.view_program['u_scaling'] = self.scaling
+        view.view_program['u_scaling'] = self._scaling_int
 
     def _compute_bounds(self, axis, view):
         pos = self._data['a_position']


Test patch: