Question: how many parameters did their model use?
Answer: Excluding the embedding weights, our model requires 100k parameters