Don't diffuse over transparent colors