You have to put the ghost traps for the ghosts to finish them. Moreover, spirits will be lost if you get too far from the ghosts.