Fix possible endless loop in inotify

Fixes #1041