differentiate bayer/xtrans demosaic methods

Previously the same method ids meant different things depending on if
the file had a Bayer or xtrans matrix. Now use a bit-mask to
differentiate the xtrans methods.