Repository: napari/napari

Issue:
Bug: values of zero in 3D MIPs render as transparent, rather than black, even in opaque blending mode
## 🐛 Bug
noticed by @maweigert:

if a volume with a bunch of zeros is in front of another volume, we would have expected that it would essentially occluded any layers behind it, assuming, of course, that the blending mode is opaque.  (in additive blending, naturally, any nonzero values in other layers would result in a non-black MIP pixel).

However, MIP values of zero appear to render as transparent, even in opaque blending mode, showing layers behind it:

```python
import numpy as np 
import napari 

x = np.zeros((20,20,20), np.float32)
x[10,10,10] = 100

y = np.zeros((20,20,20), np.float32)
y[1:-1,1:-1,1:-1]  = 100

v = napari.Viewer()
v.add_image(y)
v.add_image(x)

v.dims.ndisplay=3
```

<img width="1311" alt="Screenshot 2022-09-01 at 09 19 26" src="https://user-images.githubusercontent.com/1609449/187933993-1454960c-f9fc-4ab4-b23e-a6603e093281.png">

changing `viewer.layers['x'].rendering = 'additive'` _does_ then occlude all of `y`
not 100% sure it's a bug, but it did strike us as unexpected.  @brisvag?



Discussion:
took a look at the [mip snippets](https://github.com/vispy/vispy/blob/dd8140d6efccee75359059d7c0ed69fbad8059f1/vispy/visuals/volume.py#L367-L404) - it looks like the important thing is the value of [`u_mip_cutoff`](https://github.com/vispy/vispy/blob/main/vispy/visuals/volume.py#L1096) is, any fragments below this threshold are discarded

This is beneficial when rendering binary volumes in the image layer but I agree it's unexpected here -> we could special case binary images so both are handled appropriately or just modify the cutoff globally
side note, this has actually bitten me before too, I got invisible planes because all fragments were discarded which made it hard to interact with them... I think I'm in favour of setting the cutoff to the min of the data as a general rule
Uhm, I'm a bit confused by this... `u_mip_cutoff` is being [automatically set to the minimum of the contrast limits](https://github.com/napari/napari/blob/74fd219b59c64b58296a9ecc13a96776f255d0bc/napari/_vispy/layers/image.py#L172-L176). Shouldn't that work as intended? I feel like the real culprit here is the blending doing something slightly different from the expected :/
I think that's exactly the reason for the unexpected behaviour

Data min 0.00, min clim 0.05 -> all pixels with a max value of 0.00 after raycasting are below the cutoff? Setting the cutoff to the min of the data should lead to the behaviour Talley/Martin were expecting 

Was mip cutoff (and the min counterpart) introduced for any specific reason? Was it the 3D depth stuff? :)
Ok, I think I had misunderstood the issue. Yes, the result above is "intended", in the sense that we are discarding fragments below the contrast limits or below the minimum of the image. We don't have a way to set this *separately* from the clim. As a temporary solution, to get your desired behaviour in the example above, you can simply set the clim to `(-1, 100)`, instead of `(0, 100)`.

However, I'm not sure *why* one would want to obscure other data with blank pixels. What is the purpose here? Only thing I can think of is that you want to see the size of the raw data in relation to the rest, which to me sounds like a job for #4849.

> This is beneficial when rendering binary volumes in the image layer but I agree it's unexpected here -> we could special case binary images so both are handled appropriately or just modify the cutoff globally

I'm not following: the the `cutoff` was introduced to handle *non*-binary image (so you can set a threshold just like an isosurface). Binary volumes are basically unaffected by this cutoff.

PS: Note that the current approach of discarding fragments below a threshold is a temporary, hacky solution until we (I ^^') fix transparency in vispy and can finally assing alpha values instead of having a sharp cutoff.

PPS: I leave below some ramblings I did before realizing that I misunderstood the issue, because I think there is a bit of confusion on the role of blending:

---

Note that the above behaviour happens with `translucent_no_depth` blending on `y` and `x`, since that's the new default blending for images since #4523.

This means that depth is ignored for for `y`. This is the reason why any non-additive blending used for `x` results in `x` being overlapped to `y` (i.e: it's always in front). Additive *should* not care about depth, so the layers are blended additively. I suggest to set the colormap of `x` to `red` and `y` to cyan in order to make it clearer.

If you change the blending of `y` to `translucent`, you'll see the following behaviour when changing the blending of `x`:
a. `translucent_no_depth`: overlapped, as expected (depth of x is ignored, so only order matters)
b. `translucent`: `x` is invisible because *inside* the `y` cube, as expected
c. `additive`: layers are merged additively, as expected
d. `opaque`: `x` disappears, because `opaque` *does* account for depth.

`d` is the one I find a bit confusing (like @tlambert03 and @maweigert, I expected this to be equivalent to `translucent_no_depth` excluding transparency), but maybe I'm misunderstanding its purpose.

> However, I'm not sure why one would want to obscure other data with blank pixels. What is the purpose here? 

One reason might be if you want to toggle visibility of one volume in front of another.  You don't want to blend them, you just want to see one then the other

But more importantly, I don't think it's for napari to ask the question of what the end user wants to do here.  If one of the opaque blending methods is chosen, I would expect a black pixel to be black, not transparent 
video below is a great example of when this is useful

https://user-images.githubusercontent.com/7307488/188457126-ef540278-20eb-4f47-8a92-604d5a0a0f04.mp4

> If one of the opaque blending methods is chosen, I would expect a black pixel to be black, not transparent

This is the key insight - I think we should disable the minip/mip cutoff in opaque blending modes and leave as-is in translucent until we (you @brisvag  😆) fix transparency so we can properly composite
> One reason might be if you want to toggle visibility of one volume in front of another. You don't want to blend them, you just want to see one then the other.

Tangential, but this is exactly one of the main use cases mentioned in #4684. I don't think "visibility zapping" should be the desired to that use case, since it's only non-finnicky with the trivial use case of 1 "covering" layer. But yes, for the time being I agree that that use case should be used.

> I think we should disable the minip/mip cutoff in opaque blending modes and leave as-is in translucent

I guess that's a good solution, since currently `opaque` does not seem to meet the expectation anyways.

Resolving patch:
diff --git a/napari/_vispy/layers/image.py b/napari/_vispy/layers/image.py
--- a/napari/_vispy/layers/image.py
+++ b/napari/_vispy/layers/image.py
@@ -4,6 +4,7 @@
 from vispy.color import Colormap as VispyColormap
 from vispy.scene.node import Node
 
+from ...layers.base._base_constants import Blending
 from ...utils.translations import trans
 from ..utils.gl import fix_data_dtype, get_gl_extensions
 from ..visuals.image import Image as ImageNode
@@ -169,11 +170,29 @@ def _on_depiction_change(self):
     def _on_colormap_change(self):
         self.node.cmap = VispyColormap(*self.layer.colormap)
 
+    def _update_mip_minip_cutoff(self):
+        # discard fragments beyond contrast limits, but only with translucent blending
+        if isinstance(self.node, VolumeNode):
+            if self.layer.blending in {
+                Blending.TRANSLUCENT,
+                Blending.TRANSLUCENT_NO_DEPTH,
+            }:
+                self.node.mip_cutoff = self.node._texture.clim_normalized[0]
+                self.node.minip_cutoff = self.node._texture.clim_normalized[1]
+            else:
+                self.node.mip_cutoff = None
+                self.node.minip_cutoff = None
+
     def _on_contrast_limits_change(self):
         self.node.clim = self.layer.contrast_limits
-        if isinstance(self.node, VolumeNode):
-            self.node.mip_cutoff = self.node._texture.clim_normalized[0]
-            self.node.minip_cutoff = self.node._texture.clim_normalized[1]
+        # cutoffs must be updated after clims, so we can set them to the new values
+        self._update_mip_minip_cutoff()
+
+    def _on_blending_change(self):
+        super()._on_blending_change()
+        # cutoffs must be updated after blending, so we can know if
+        # the new blending is a translucent one
+        self._update_mip_minip_cutoff()
 
     def _on_gamma_change(self):
         if len(self.node.shared_program.frag._set_items) > 0:


Test patch: