DOI: 10.5061/dryad.ht76hdrpf
title: Data from: Neural dynamics of the processing of speech features: Evidence for a progression of features from acoustic to sentential processing

description: When we listen to speech, our brain’s neurophysiological responses “track”
 its acoustic features, but it is less well understood how these auditory
 responses are enhanced by linguistic content. We recorded
 magnetoencephalography (MEG) responses while subjects listened to four
 types of continuous-speech-like passages: speech-envelope modulated noise,
 English-like non-words, scrambled words, and a narrative passage. Temporal
 response function (TRF) analysis provides strong neural evidence for the
 emergent features of speech processing in cortex, from acoustics to
 higher-level linguistics, as incremental steps in neural speech
 processing. Critically, we show a stepwise hierarchical progression of
 progressively higher order features over time, reflected in both bottom-up
 (early) and top-down (late) processing stages. Linguistically driven
 top-down mechanisms take the form of late N400-like responses, suggesting
 a central role of predictive coding mechanisms at multiple levels. As
 expected, the neural processing of lower-level acoustic feature responses
 is bilateral or right lateralized, with left lateralization emerging only
 for lexical-semantic features. Finally, our results identify potential
 neural markers, linguistic level late responses, derived from TRF
 components modulated by linguistic content, suggesting that these markers
 are indicative of speech comprehension rather than mere speech
 perception.  This dataset includes raw MEG
 (magnetoencephalography) data, behavioral responses, stimuli, predictors,
 main codes, some intermediate results (Temporal response functions (TRFs),
 features extracted from TRFs), and statistical analysis codes related to
 the above findings, paper titled "Neural Dynamics of the processing
 of speech features: Evidence for a Progression of Features from Acoustic
 to Sentential Processing"
description: Magnetoencephalography (MEG) data were recorded from young adult
 participants as they listened to four types of continuous-speech-like
 passages: speech-envelope modulated noise, English-like non-words,
 scrambled words, and narrative passage. All information related to
 experimental procedure, stimuli, preprocessing, and data anlaysis are
 described in the paper.
description: # Neural Dynamics of the processing of speech features: Evidence for a
 Progression of Features from Acoustic to Sentential Processing
 [https://doi.org/10.5061/dryad.ht76hdrpf](https://doi.org/10.5061/dryad.ht76hdrpf) The four passage types used in the study are abbreviated as follows. * ASM - Scrambled word passage * ASN - Speech modulated noise passages * PH - Nonword passages * AS - Story passage ## Description of the data and file structure ### **stimuli.zip** 1. wav files - stimuli (.wav) used in the study. 2. transcripts - transcripts of .wav files in 'wav files' folder 3. acoustic comparisons - Acoustic comparison analysis and results plotted in Figure S5 ### **predictors.zip** predictors (Gammatone envelope (gammatonelog-8), gammatone envelope onset (gammatonelog-on-8), phoneme onsets (pho), word onsets (wor), phoneme surprisal (psr1), cohort entropy (pen), unigram word surprisal (wfr) and GPT2 word surprisal (hug)) used for TRF modeling. Each file contains the speech features their amplitudes and latencies. Naming convention is `[passagetype]_M0[stimulinumber]|[predictor].pickle` ### **meg.zip** Each folder contains the meg data for each subject and emptyroom measurements. All files are .fiff files. The .fiff files are data recorded from MEG KIT at the University of Maryland College Park ([https://linguistics.umd.edu/resources-facilities/labs/KIT-Maryland-MEG-Lab](https://linguistics.umd.edu/resources-facilities/labs/KIT-Maryland-MEG-Lab)). The data can be viewed in python using mne python package ([https://mne.tools/stable/generated/mne.io.show_fiff.html)](https://mne.tools/stable/generated/mne.io.show_fiff.html\)) ### **Codes.zip** Main codes (experiment_class using Eelbrain ([https://eelbrain.readthedocs.io/en/stable/reference.html#module-pipeline](https://eelbrain.readthedocs.io/en/stable/reference.html#module-pipeline)) (exp.py), and Jupyter notebooks for TRF computation, TRF feature extraction, and prediction accuracy comparisons. Additionally, this folder includes statistical analysis codes and results figures 1\. **exp.py** sets up the experiment pipeline for this specific dataset along with other pre-processing options, TRF modeling, epoch extraction and parcellation. 2\. **figure2** - Results figures for Fig 2, generated from 'prediction accuracy.ipynb' 3\. **prediction accuracy.html** - 'prediction accuracy.ipynb' saved as .html 4\. **prediction accuracy.ipynb** - Once the exp.py is setup properly, 'prediction accuracy.ipynb' can be run. 'prediction accuracy.ipynb' gives example codes for prediction accuracy comparisons in Table S1. 5\. **R analysis** - R codes used for statistical analysis, plotting, and predictor comparisons between passage types 6\. **TRFs** - Includes the features extracted from Temporal Response Functions (TRFs) (P1 : early positive polarity peak and N1 : Late negative polarity peak, peak amplitudes (normalized magnitudes, arbitrary unit) and latencies in (ms)) for each subject, brain hemisphere, speech condition, and speech feature. Naming convention for .csv files are `[speech feature]_[hemi]ftp.csv`. Where speech feature is either (Gammatone envelope (ENV), gammatone envelope onset (ENVON), phoneme onsets (PHO), word onsets (wor), phoneme surprisal (PSR), cohort entropy (PEN), unigram word surprisal (WFR) and GPT2 word surprisal (HUG)), and hemi is either (left (lh) or right (rh)) The variables in tables (.csv) files are as follows subject = Subject ID * \[passagetype] = Includes the TRFs * \[passagetype]_P1 = amplitude of early peak (arbitrary unit) * \[passagetype]_P2 = amplitude of middle peak (arbitrary unit) * \[passagetype]_N1 = amplitude of late peak (arbitrary unit) * \[passagetype]_*P1*_latency = Latency of early peak (ms) * \[passagetype]_*P2*_latency = Latency of middle peak (ms) * \[passagetype]_*N1*_latency = Latency of late peak (ms) * mask = ROIS (ftp = Frontal Temporal Parietal) * hemi = hemisphere (rh=right hemisphere, lh = left hemisphere) 'figs' folder includes the figures layered in Fig 3,4,5 and Figures S1, S2, S3. Each folder within fig includes the figures for each speech feature (ENV, ENVON, HUG,PEN, PHO,PSR,WFR, and WOR) The naming convention for .eps files inside each of those folder is `[speech feature]_[hemi]ftp_[peak].eps`. Where speech feature and hemi are same as defined above and peak is either (early (P1), middle (P2) or late (N1)) peak_latency.eps includes the figure for Figure 6B in the paper 7\. TRFs example.html - 'TRFs example.ipynb' saved as .html 8\. TRFs example.ipynb - Example code to extract peak amplitudes and latencies and from unigram word surprisal TRFs and comparison between narrative and scrambled passages. 9\. TRFs_main.py - TRF peak extraction code definition ### **behavioral.zip** Behavioral responses during the data recording. The variables in tables are as follows 1. MEGID = Subject ID 2. Age = Age in years 3. Gender = Male (M) / Female (F) 4. Handedness = Right (R) / Left (L) 5. All other columns are the questions associated with each passage in the stimuli set. If it is a question ex. 'what was the fellow in the canoe doing? Responses right/ wrong/ did not answer are marked as R/W/did not answer respectively. If it is a `[word(number)]` ex. LELEK(1), it represents the probe word asked and the actual number of counts in the passage. Responses show the answers (numbers) given by the participants.  ### **TRFs.zip** Estimated TRFs for different models using boosting algorithm ([https://eelbrain.readthedocs.io/en/stable/generated/eelbrain.boosting.html](https://eelbrain.readthedocs.io/en/stable/generated/eelbrain.boosting.html)) for each subject and passage type. The .pkl files include the boosting result for each subject. The file naming format is '{subjects}/*{passage type} nobl - 120-900 100 Hz ftp model {model number} boosting h50 l1 4ptns ss1 cv.pkl'., where Model117 = gammatonelog-8 + gammatonelog-on-8 + pho + wor + psr1 + pen + wfr + hug, Model 111 = gammatonelog-8 + gammatonelog-on-8 + pho + wor + psr1 + pen, and Model 116 = gammatonelog-8 + gammatonelog-on-8 + pho + wor + psr1 + pen + wfr ## Code/Software Require python packages are - eelbrain, mne, and trftools Required R packages are - ggplot2, dplyr, tidyr, readr, ggpubr