[#54854900] Properly handle relative paths for JSON output

@elainethale this has all of my unit tests passing. I'm getting vagrant
set up and will be able to test with it soon.