As of now, when the image is 3 channel, we are considering source depth. Only for 1 channel images we are considering 8U depth. And if the depth is anything other than 8U, we wouldn't be able to display it using pygame. It would be possible with our own rendering utility which converts it to 8U to show. So, I think we should keep 8U for 1.3 and/or 1.4. And when we are able to support other depths, we should allow it.

And the test which it broke was regarding segmentation which created a 32F image. I think we have to support that. Filters, DFT image, etc will have 32F or 64F depth and we also need to support that.

So, I think we should support all depths, but display utility will support only 8U, So maybe add a warning when user/dev trying to show image of depth other than 8U. If we do this, does it mean that we support all image depths? or just 8U?