Is second shuffle necessary?  Couldn't the mask_ids just be monotonically increasing?

Original name -> answer map: 
choice_0 -> A
choice_1 -> B
choice_2 -> C
choice_3 -> D

After shuffle (secondary mapping stored in self.mask_dict):
masked_choice_0 -> choice_2 -> C
masked_choice_1 -> choice_0 -> A
masked_choice_2 -> choice_3 -> D
masked_choice_3 -> choice_1 -> B