Implement support for max num ref pts and translation only