Question: what is the word level baseline they compare to?
Answer: a simple word-level encoder, with the input as words instead of characters
Alternative answer: The encoder is essentially the same as tweet2vec, with the input as words instead of characters