Handle deletion on watch dirs