text: 52,20,1333498158853222400,846245360,Alex Tamkin,"What happens when you Fourier Transform a BERT neuron? Signal processing can reveal (+manipulate!) multiscale linguistic structure in BERT neurons! New #NeurIPS2020 paper w/ @jurafsky and Noah Goodman  Paper: <LINK> @stanfordnlp & @StanfordAILab 👇 1/ <LINK> Linguistic phenomena occur at different scales, including – within a word (e.g. morphology) – between words (syntax) – across utterances (discourse) – across paragraphs (topic) But to what extent are these scales captured in the representations of pretrained models? 2/ <LINK> It turns out that many BERT neurons exhibit *multiscale* structure across an input—this neuron changes both rapidly between adjacent tokens and gradually across the input Spectral analysis lets us disentangle these scales by treating these activations as a digital signal! 3/ <LINK> After doing this, we show through probing experiments that different parts of a neuron's frequency spectrum capture knowledge of NLP tasks at different scales!  Low frequencies correspond to topic, higher frequencies to part of speech, and middle ones to dialog acts. 4/ <LINK> We can also use spectral filters to *specialize* neurons to different scales of structure during pretraining, using what we call a prism layer. This produces single multiscale representations which perform comparably or better on all tasks. 5/ <LINK> The prism layer also improves modeling of long-range dependencies—the BERT + Prism model outperforms BERT on masked language modeling problems without local context! 6/ <LINK> Spectral filters are super easy to incorporate into existing models—it just takes a couple lines of code and an off-the-shelf PyTorch library (<LINK>) 7/ <LINK> It was great working on this with @jurafsky and Noah Goodman, and we're excited to see what folks do with these methods, both for interpretability and in building better models! Paper: <LINK> 🌇 8/8",https://arxiv.org/abs/2011.04823,"Language exhibits structure at different scales, ranging from subwords to words, sentences, paragraphs, and documents. To what extent do deep models capture information at these scales, and can we force them to better capture structure across this hierarchy? We approach this question by focusing on individual neurons, analyzing the behavior of their activations at different timescales. We show that signal processing provides a natural framework for separating structure across scales, enabling us to 1) disentangle scale-specific information in existing embeddings and 2) train models to learn more about particular scales. Concretely, we apply spectral filters to the activations of a neuron across an input, producing filtered embeddings that perform well on part of speech tagging (word-level), dialog speech acts classification (utterance-level), or topic classification (document-level), while performing poorly on the other tasks. We also present a prism layer for training models, which uses spectral filters to constrain different neurons to model structure at different scales. Our proposed BERT + Prism model can better predict masked tokens using long-range context and produces multiscale representations that perform better at utterance- and document-level tasks. Our methods are general and readily applicable to other domains besides language, such as images, audio, and video. ","Language Through a Prism: A Spectral Approach for Multiscale Language