Check directory matches when caching dataset