Fix reference set calculations for decode & encode