Re-jiggle the paths so extension.json can be auto-generated