Definition: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
Input: The problem with morpheme segmentation is that the vocabulary of stem units is still very large, which leads to many rare and unknown words at the training time. Therefore, we learn a BPE model on the stem units in the training corpus rather than the words, and then apply it on the stem unit of each word after morpheme segmentation.
Question: How is morphology knowledge implemented in the method?
Output:
A BPE model is applied to the stem after morpheme segmentation.