Make setting the primitive map conditional, because it helps the trace optimizer

Signed-off-by: Stefan Marr <git@stefan-marr.de>