Inference time on MMLU
#60
by
kbganesh
- opened
Does anyone know approximately how long the model is expected to take for inference on the MMLU benchmark (5-shot setting)? I am currently using 8 V100-32GB GPUs, and I'm using the Huggingface/transformers library, and it takes me approximately 7 hours for inference (with no optimizations like TensorRT), and I am not sure if this is expected.