If frames are skipped, try to make up for it by back scheduling