Fix incorrect frame order, change frame index to uint

Frame index was handled incorrectly. It was an int but assumed to roll
over. It wasn't ensured that frameBufferCount is always a power of two,
which could have caused errors once the counter hit INT_MAX.