Do not duplicate work in SpecConstr (fix Trac #7865)

This is a bad bug, if a rare one.
See Note [Work-free values only in environment].

Thanks to Amos Robinson for finding it.