Factor out build-inverse-map