Ignore cached data when fetching refs

Apparently these get cached for 60s, which can result in stale data
when performing git operations.