  This paper describes a system submitted by team BigGreen to LCP 2021 for
predicting the lexical complexity of English words in a given context. We
assemble a feature engineering-based model with a deep neural network model
founded on BERT. While BERT itself performs competitively, our feature
engineering-based model helps in extreme cases, eg. separating instances of
easy and neutral difficulty. Our handcrafted features comprise a breadth of
lexical, semantic, syntactic, and novel phonological measures. Visualizations
of BERT attention maps offer insight into potential features that Transformers
models may learn when fine-tuned for lexical complexity prediction. Our
ensembled predictions score reasonably well for the single word subtask, and we
demonstrate how they can be harnessed to perform well on the multi word
expression subtask too.
