Set amask based on ReplaceAlphaWithStencil()