ability to build crawler storage from FS; for offline testing, see comment in starter