Hierarchical Frequency Tagging Probe (HFTP): A Unified Approach to
  Investigate Syntactic Structure Representations in Large Language Models and
  the Human Brain
Authors: name: Jingmin An, name: Yilong Song, name: Ruolin Yang, name: Nai Ding, name: Lingxi Lu, name: Yuxuan Wang, name: Wei Wang, name: Chu Zhuang, name: Qian Wang, name: Fang Fang

Large Language Models (LLMs) demonstrate human-level or even superior
language abilities, effectively modeling syntactic structures, yet the specific
computational modules responsible remain unclear. A key question is whether LLM
behavioral capabilities stem from mechanisms akin to those in the human brain.
To address these questions, we introduce the Hierarchical Frequency Tagging
Probe (HFTP), a tool that utilizes frequency-domain analysis to identify
neuron-wise components of LLMs (e.g., individual Multilayer Perceptron (MLP)
neurons) and cortical regions (via intracranial recordings) encoding syntactic
structures. Our results show that models such as GPT-2, Gemma, Gemma 2, Llama
2, Llama 3.1, and GLM-4 process syntax in analogous layers, while the human
brain relies on distinct cortical regions for different syntactic levels.
Representational similarity analysis reveals a stronger alignment between LLM
representations and the left hemisphere of the brain (dominant in language
processing). Notably, upgraded models exhibit divergent trends: Gemma 2 shows
greater brain similarity than Gemma, while Llama 3.1 shows less alignment with
the brain compared to Llama 2. These findings offer new insights into the
interpretability of LLM behavioral improvements, raising questions about
whether these advancements are driven by human-like or non-human-like
mechanisms, and establish HFTP as a valuable tool bridging computational
linguistics and cognitive neuroscience. This project is available at
https://github.com/LilTiger/HFTP.