Only poll the filesystem when necessary

Fixes #96.