Invalidate local sources on refresh

Refresh should always invalidate local sources. Lost in save/refresh split.

cc @samanpwbb