Allow setting the frequency of which the file watcher checks for updates

Allow setting the watcher to only check for update once every request (default), every time (mostly useful for tests) or never (if we know the underlying filesystem doesn't change anyway)

@DeepDiver1975  as discussed.

cc @PVince81