Before revision:  We explore representations from different model families (BERT, RoBERTa, GPT-2 , etc.) and find evidence for emergence of linguistic manifold across layer depth (e.g., manifolds for part-of-speech and combinatory categorical grammar tags).
After revision:  We explore representations from different model families (BERT, RoBERTa, GPT-2 , etc.) and find evidence for emergence of linguistic manifolds across layer depth (e.g., manifolds for part-of-speech and combinatory categorical grammar tags).