swap buffer only on CastSwapBuffers

Fixes #34.