fix a refresh issue

Remove an assert that was triggered when switching to and from applications quickly. Changed our semantics to only only one refresh at a time, but not throw an assert about it.