Use path.join instead of path.resolve

This ensures the case of the path is consistent with how require resolves
paths.

Closes #2486