Fix regression causing setup to run for every play

This patch makes sure setup only runs once, unless it is enforced on a play.