Fix regression causing setup to run for every play

This patch makes sure setup only runs once, unless it is enforce on a play.