Fix restrict_universe_context to not remove useful constraints.