Make sure autoplay test callback is only called once

Closes #1279