Use correctly calculated PTS offset.
libav only set the time_base of streams after it has written the file header