Only show upgrade notices, and use transient cache

Closes #5211