Don't compute the visibility unless we really have to.

This brings the number of linkage computations in "clang -cc1" in SemaExpr.ii
from 58426 to 43134. With -emit-llvm the number goes from 161045 to 145461.

git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@182823 91177308-0d34-0410-b5e6-96231b3b80d8