What you can cram into a single \$&!#* vector: Probing sentence embeddings for linguistic properties
Probing for semantic evidence of composition by means of simple classification tasks
Looking for ELMo's friends: Sentence-Level Pretraining Beyond Language Modeling
Sentence Ordering and Coherence Modeling using Recurrent Neural Networks
Illustrative Language Understanding: Large-Scale Visual Grounding with Image Search
Dissecting Contextual Word Embeddings: Architecture and Representation