Do lookups of relative paths and memoize results