The balance mechanism in our inner ears is one system we have, but visual clues are also important and can be overriding. If the horizon cannot be seen or is not level then we may be fooled by objects which we expect to be vertical but which aren't really. False perspective may also play a role. If a line of trees get larger or smaller with distance away, our sense of perspective is thrown off. Objects far away may seem smaller or larger than they really are.
People often overestimate the angle of a slope.
If you are standing on a slope of 1 degrees it will seem like a slope of 5 degrees and if you stand on a slope of 5 degrees it may seem like you are on a slope of 30 degrees. Because of this effect the anti-gravity illusion can seem stronger than it should be even when you know the cause. Even when the true cause is understood it can be difficult to believe. In some cases the sea horizon is partly visible and it seems incredible that the effect can be an illusion. If you think there is a magnetic anomaly just use two plumb lines, one made of iron and one of stone.