  Small perturbations in the dielectric environment around a high quality
whispering gallery mode resonator usually lead to a frequency shift of the
resonator modes directly proportional to the polarizability of the
perturbation. Here, we report experimental observations of strong frequency
shifts that can be opposite and even exceed the contribution of the
perturbations' polarizability. The mode frequencies of a lithium niobate
whispering gallery mode resonator are shifted using substrates of refractive
indices ranging from 1.50 to 4.22. Both blue- and red-shifts are observed, as
well as an increase in mode linewidth, when substrates are moved into the
evanescent field of the whispering gallery mode. We compare the experimental
results to a theoretical model by Foreman et al. and provide an additional
intuitive explanation based on the Goos-H\"anchen shift for the optical domain.
