Fixed event handling to handle case where we delete/rename pubspec.yaml or cache

Fixed #3018

review: @devoncarew

We could remove most (all) of this special case handling for `cache` here.