Fixed issue with refreshing

Cache is now only rebuilt on the following:
- First time
- New sites exist
- Manual refresh from context