Fix handling of RGBA buffers in CalRGB colorspace (issue 5270)

Fixes #5270.

*Note:* The file in the issue is only available through a file hosting service, hence I didn't include it as a linked test because of the (fairly large) risk that it becomes unavailable in the future.