In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.

[EX Q]: We show that by determining and integrating heterogeneous set of features from different modalities aesthetic features from posted images (colorfulness, hue variance, sharpness, brightness, blurriness, naturalness), choice of profile picture (for gender, age, and facial expression), the screen name, the language features from both textual content and profile's description (n-gram, emotion, sentiment), and finally sociability from ego-network, and user engagement we can reliably detect likely depressed individuals in a data set of 8,770 human-annotated Twitter users.
Question: What is the source of the textual data?
[EX A]: Users' tweets

[EX Q]: We train a single layer LSTM with a 150-dimensional hidden state for hate / not hate classification. The input dimensionality is set to 100 and GloVe BIBREF26 embeddings are used as word input representations.
Question: What unimodal detection models were used?
[EX A]: single layer LSTM with a 150-dimensional hidden state for hate / not hate classification

[EX Q]: However, a significant bottleneck that has been overlooked by previous efforts is the input vocabulary size and its corresponding word embedding matrix, often accounting for a significant proportion of all model parameters. For instance, the embedding table of the BERTBASE model, comprising over 30K WordPiece tokens BIBREF14, accounts for over $21\%$ of the model size. While there has been existing work on reducing NLP model vocabulary sizes BIBREF15, distillation techniques cannot utilize these, since they require the student and teacher models to share the same vocabulary and output space. This profoundly limits their potential to further reduce model sizes.
Question: Why are prior knowledge distillation techniques models are ineffective in producing student models with vocabularies different from the original teacher models?
[EX A]:
While there has been existing work on reducing NLP model vocabulary sizes BIBREF15, distillation techniques cannot utilize these, since they require the student and teacher models to share the same vocabulary and output space. This profoundly limits their potential to further reduce model sizes.