Task: Question Answering
Input (English): We varied the number of experts between models, using ordinary MoE layers with 4, 32 and 256 experts and hierarchical MoE layers with 256, 1024 and 4096 experts. 
 Question: How is the correct number of experts to use decided?
Output (English): varied the number of experts between models