Add a delay before really removing the source information in case the file is about to be readded

(this happens when you do 'git stash pop' for instance)