added crawler to automatically add files from parent directories