From: kieffer@spf.trw.com (Robert Kieffer)
Subject: Re: Changing alpha only?
Organization: TRW Inc., Redondo Beach, CA
Date: Tue, 10 Nov 92 16:49:28 GMT

In article <1992Nov10.053919.19601@ohsu.edu> filibert@vista (Daniel  
Filiberti) writes:
> Hi Everybody!
> 
> Another quirky problem.  I need to mask an area of an image
> by changing the alpha value within the area to 1 (for
> example) and everything outside the area to (0).  The
> catch is that I need to retain the original RGB data within
> the whole image (not just the masked area! otherwise
> clipping would work!).  I have a path to clip to, easy
> enough, but how can I draw only to the alpha channel?  Or,
> how do I use some strange combination of compositing
> functions to do this?  The images are RGB tiffs, no alpha,
> so compositing defaults to an opaque alpha (1, I
> think)... 
> 
> In simpler terms, I need all of the image data, and the
> alpha channel to mask a region of interest within
> it...easy, right?  Any takers? 
> 
> Daniel Filiberti 

Well Daniel, I'm afraid you're out o' luck.

What you were probably looking for is the DIN ("destination in") composite  
operation.  The output is the destination data, masked by the source alpha  
value.  However, you're going to run into problems I expect.  Read on...

I ran into a problem trying to do some related work a few months back  
(i.e. change alpha w/out affecting the RGB values). So, I started  
exploring...

It turns out that NeXT kind of outsmarts themselves here.  In their (TIFF)  
representation of image data, they do a pre-multiplication of the RGB  
channels by the alpha channel to help speed up the compositing operations.   
This comes from the fact that the SOVER operation, for instance, is  
mathematically represented as:

	RGB' = a * RGB1 + (1-a) * RGB2

	RGB' = new value for red, green, or blue channel
	RGB1 = overlay(source) RGB value
	RGB2 = "underlay" (destination) RGB value
	a = alpha value of overlay image
	(all values range 0 -> 1)

What Next does is store the RGB values already multiplied by the  
corresponding alpha (i.e. the "a*RGB1" component of the above equation).   
Now then, keeping in mind that the values used in the 8-bit TIFF images  
represent "virtual" values of 0 -> 1, any RGB value that is larger then  
the alpha value is invalid. So, if you use the composite operation I  
described above to make a region of your image totally transparent (set  
the alpha to 0) all corresponding color information also gets set to zero  
and is subsequently lost! :^(  Returning the alpha to 1 (255 in 8-bit)  
just multiplies the, now zero, color information by one and you get zero,  
a black image.

The upshot of this is that if you want to retain the RGB information,  
you're better off keeping a raw version of the data around, and tweaking  
with the alpha values separately... then, when you want to display it, do  
the multiplication by alpha and create a [disposable] NXBitmapImageRep for  
it.

Sorry, this got kind of long winded.  I hope it makes sense.

I vaguely recall confirming all of this with Ali Ozer, the NeXT graphics  
guru, at the time I was hashing it out so I'm pretty sure it's correct.   
It shouldn't be hard to write your own routine to check this out if you  
want, though.

Anyway, let me know if you have questions.

Robert Kieffer
kieffer@spf.trw.com
Newsgroups: comp.sys.next.programmer