For ENG-5962, fix a bug maintainingg metadata for partition info, remove some indirection and improve locality