If it was, all samples of Vermilion would have blackened. It doesn't out-gas like orpiment and realgar, and it's not an unstable compound like the copper greens. Varnish should not make a bit of difference with Vermilion. Only certain pigments like Titanium white are truly perfectly stable and can be mixed with anything. If perfect permanence was the only characteristic of a pigment that is valuable, we would all be painting with Titanium white instead of Flake, and using Cobalt or Cerulean Blue rather than any organic blue.
Reasonable permanence is acceptable.
The same thing is often said about cadmium colors, too. This is because Cadmiums and Vermilion are very similar chemically. They are both metallic sulfides. The free sulfur or polluting sulfides may react with lead white. Many people strongly warn against mixing cadmiums with lead white.