Title: BriLLM: Brain-inspired Large Language Model
Venue: NeurIPS 2025
Accepted: no
Keywords: LLM
Abstract: This paper reports the brain-inspired large language model (BriLLM). This is a non-Transformer, non-GPT, non-traditional machine learning input-output controlled generative language model. The model is based on the Signal Fully-connected flowing (SiFu) definition on the directed graph in terms of the neural network, and has the interpretability of all nodes on the graph of the whole model, instead of the traditional machine learning model that only has limited interpretability at the input and output ends. In the language model scenario, the token is defined as a node in the graph. A randomly shaped or user-defined signal flow flows between nodes on the principle of "least resistance" along paths. The next token or node to be predicted or generated is the target of the signal flow. As a language model, BriLLM theoretically supports infinitely long $n$-gram models when the model size is independent of the input and predicted length of the model. The model's working signal flow provides the possibility of recall activation and innate multi-modal support similar to the cognitive patterns of the human brain. At present, we released the first BriLLM versions in Chinese and English, with 4000 tokens, 32-dimensional node size, 32-token sequence prediction ability, model sizes around 2B and 1B respectively, bringing language model prediction performance comparable to GPT-1.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces BriLLM, a novel LLM inspired by the biological signal propagation mechanisms of the human brain. Instead of using the dominant Transformer acchitecture, BriLLM is constructed on a graph-based framework called Signal Fully-connected Flowing (SiFu). SiFu represents tokens as nodes in a fully connected directed graph. Each edge in the graph is a learnable fully-connected neural network and each token in the vocabulary is modeled as a node. The generation process is guided by an \"energy tensor\" that flows between nodes along paths of least resistance. The author propose the Chinese and English version of BriLLM and provide several empirical results on language modeling tasks.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. Novel architecture: BriLLM utilize a biologically motivated signal flow framework instead of traditional attention mechanism to build an LLM. This design enables token-level interpretability throughout the model.\n2. Efficient sparse training strategy: Though the original BriLLM has a large parameter count, the authors introduce a sparse training strategy that dramatically reduces effective model size by freezing connections associated with low-frequency token pairs.\n\n**Weaknesses**\n1. Limited experiments: The experiments are insufficient to demonstrate the viability of BriLLM. The results are largely qualitative (decoding examples), and no rigorous benchmark comparisons (e.g., on perplexity or downstream tasks) are provided. These experiments are insufficient to support the authors' claim that BriLLM's performance is comparable to GPT-1.\n2. Unsatisfactory generation quality: In the limited qualitative experiments presented in the paper, BriLLM's performance is also not good enough. While some decoding results on the training set can be viewed as grammatically plausible sentences, its outputs on the test set are entirely confusing and incoherent.\n3. Computational complexity: Compared to Transformer architectures with time complexity of $\\mathcal{O}(L^2 \\cdot d)$, where $L$ is the sequence length and $d$ the hidden size, BriLLM’s SiFu mechanism incurs a significantly higher computational cost of $\\mathcal{O}(L \\cdot n \\cdot d^2)$, where $n$ is the vocabulary size and $d$ the node dimensionality. For example, with typical settings ($n=4000$, $d=32$, $L=32$), BriLLM requires approximately $131$ million operations per sequence, whereas a Transformer under the same sequence length and $d=1024$ requires only about $1$ million operations. This means that BriLLM can incur over $100\\times$ the computation of a comparable Transformer. \n4. Inadequate references: The paper includes only two references—Transformer and GPT-1—which is far below the norm for academic publications. It fails to cite related work in areas like alternative architectures, energy-based models and graph neural networks.\n\nQUESTIONS:\n1.  Can the authors provide quantitative results (e.g., perplexity) on standard language modeling benchmarks (e.g., WikiText-103[1])? \n2.  Can the authors more precisely define the “energy tensor” and explain how it differs from standard activations or attention scores?\n3. Why are other non-Transformer and biologically inspired models (e.g., Hebbian learning, graph-based LMs, or energy-based models) not discussed or cited?\n\n[1] Merity, Stephen, et al. \"Pointer Sentinel Mixture Models.\" International Conference on Learning Representations. 2017.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 1, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-23"}

comments: SUMMARY:
To overcome Transformers’ interpretability shortcomings, the paper introduces a brain-inspired language-model design that treats every vocabulary token as a graph node and frames generation as energy flowing along weighted edges, giving node-level interpretability and sequence-length independence.

STRENGTHS AND WEAKNESSES:
Strengths 

This is an interesting and newly proposed architecture for interpretability inspired by brain.

Weaknesses
1) Evaluation is restricted to 100M token Wikipedia subsets (≤32 tokens per sample); the paper reports qualitative completions but no perplexity or downstream benchmarks, so competitiveness is unclear.

2) The authors claim their released models perform on par with GPT-1, yet they present no supporting downstream evaluation results.

3) Another key limitation is scalability: the parameter count grows quadratically with vocabulary size (O(n²)), making BriLLM impractically large for realistic token sets.

QUESTIONS:
1) Can the author provide evaluation on some downstream benchmark tasks?
2) Can the authors show results on different scales of models to evaluate whether their proposed approach will scale or not?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 1
ORIGINALITY: 2
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents BriLLM, a brain-inspired language model that replaces attention mechanisms with a novel Signal Fully-connected Flowing (SiFu) framework. Tokens are modeled as nodes in a fully connected graph, and language generation is achieved by simulating signal propagation along energy-maximizing paths. This design allows for non-autoregressive decoding, full model interpretability, and sequence-length-independent modeling. The authors implement Chinese and English versions, achieving GPT-1-level performance with significantly fewer parameters via sparse training.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nBriLLM introduces a brain-inspired, non-Transformer architecture using signal propagation over a fully connected graph (SiFu), offering an original direction in language modeling. Unlike attention-based models, BriLLM provides full node-level interpretability, potentially aiding transparency and multi-modal extensibility.\n\n**Weaknesses**\n\n* Limited empirical support: The experiments are small in scale, conducted on short sequences and small vocabularies, with performance only comparable to GPT-1 despite larger model sizes. No downstream tasks or fine-tuning are explored.\n* Clarity and exposition issues: Key concepts such as \"signal energy,\" training procedures, and decoding paths are described vaguely, with unclear terminology and underexplained figures, making the method hard to follow.\n* Lack of theoretical and comparative depth: The paper offers no formal analysis of the SiFu mechanism, its training dynamics, or convergence properties. Additionally, the reported loss curve exhibits signs of overfitting, raising concerns about generalization. The work also lacks a thorough discussion of related literature, particularly regarding energy-based models and graph neural networks, making it difficult to assess its novelty in context.\n* Scalability concerns: The fully connected nature of the model leads to dramatic parameter growth and costly training, limiting practicality even with sparsification techniques.\n* Presentation issues also detract from the overall clarity. Figures 2 and 3 are oversized relative to their informational content, Figure 4 is visually unpolished, and Tables 2 and 3 occupy excessive space while adding limited value. These formatting inconsistencies, combined with the relatively sparse core content, make the paper feel underdeveloped and less polished than expected for the submission.\n\nQUESTIONS:\n1. Is the proposed method empirically convincing given that it is only tested on small-scale datasets with limited sequence length and achieves performance comparable to GPT-1 despite a much larger model size?\n\n2. Are the core concepts and training procedures sufficiently clear and well-explained to allow readers to fully understand and reproduce the method?\n\n3. Does the paper provide adequate theoretical grounding and related work comparison to justify the novelty and validity of the proposed SiFu mechanism?\n\n4. Is the model design practically scalable, considering its fully connected structure leads to substantial parameter growth and high computational cost?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces BriLLM, a brain-inspired large language model that deviates from the conventional neural architectures like Transformer. Each label in the vocabulary corresponds to a node in a directed graph. There is one node active at a time. The current \"signal\" (in the current node) is a vector of shape [d_{node}]. Calculating the next signal when going from node u to v is via a linear projection.\n\nThere are no benchmarks, nor any comparison to other literature.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- Released code and models publicly.\n\n- Interesting new model.\n\n\nWeaknesses:\n\n- No benchmarks. (This alone is already reason for strong reject.)\n\n- No analysis.\n\n- No comparison to related work.\n\n- Many parts unclear.\n\nSee my questions/comments below.\n\nQUESTIONS:\nAbstract \"Signal Fully-connected flowing (SiFu) definition on the directed graph\" - the abstract should be easy to understand for the target audience of the paper, which are people familiar with language models, Transformers, maybe LSTM, Mamba, and other recent neural network types. I have no idea what SiFu is, and I assume the same for most of the readers. So this is bad. Later in the paper, it becomes clear that SiFu is what you actually propose here. In the abstract, it sounds as if you base your work on SiFu, i.e. that SiFu is some other existing concept, and you build upon it. This is very different. You should fix the wording in the abstract, and say that this is what you propose.\n\nAbstract \"In the language model scenario, the token is defined as a node in the graph\" - \"token\" is not clear here. I assume you mean subword (label)? Make this clear.\n\nAbstract \"The model’s working signal flow provides the possibility of recall activation and innate multi-modal support similar to the cognitive patterns of the human brain.\" - I don't understand this sentence. What does this mean? \"recall activation\"? \"innate multi-modal support\"? \"cognitive patterns of the human brain\"? Is this similarity actually measured somehow, or is this just a descriptive statement to give a better intuition of the model? It has the negative effect: I'm very confused by it.\n\nAbstract \"32-dimensional node size\" - I don't know what this is supposed to mean.\n\nAbstract \"32-token sequence prediction ability\" - again I'm not sure what this means.\n\nAbstract \"model sizes around 2B and 1B respectively\" - parameters? Or what?\n\nAbstract \"language model prediction performance comparable to GPT-1\" - GPT1 has just 117M params, so it's much smaller? So the presented architecture seems to perform very badly then? This statement is a bit confusing.\n\nIntro \"However, even SOTA LLMs are still in terms of Transformer architecture and GPT training scheme unlikely to laugh at the final termination of AGI ...\" - I failed to parse this sentence. \"unlikely to laugh\" - what?\n\nSec 2 \"Inspired by the working mode of the brain\" - It's not clear at all what you mean by that. The \"working mode of the brain\" can mean anything. You either need to be more specific, explain what aspect of the brain is the inspiration, also provide some references to that. Or you should better remove such a statement, if you don't really know good references for it, and if this is just your imagination of how the brain might work.\n\nFig 1b caption: \"Optimization\" - what is optimized? Does this refer to training the model, i.e. optimizing (minimizing) some loss function w.r.t. the parameters? Then say \"Training\" or so instead, or just be more specific what you mean.\n\nNote on Latex math: Don't write `d_{node}`. Write `d_{\\textrm{node}}` or so.\n\nNotation: You use u and v both for nodes and for the token sequence. I think this is confusing. As I understand, the tokens/labels/classes are exactly the nodes? There are no other nodes? Still, this is confusing.\n\nSo, you have n nodes (because n is the vocab size)? (\"n\" should already be introduced in Sec 3.)\nAll nodes are connected with each other? (This should be clearly stated also in Sec 3.) So it means you have n^2 * d_{node}^2 params for the weight matrices? (Ah yes, later in Sec 4, you also write that... But this should be explained already earlier in Sec 3, to better understand it.)\n\nSec 3: You introduce $\\mathcal{E}_i$\nin the text.\n\nbut then in eq 2 / eq 3, you define\n$\\mathcal{E}_{L-1}$ ?\nHow is $\\mathcal{E}_i$ defined?\n\nSec 3 \"learnable weights $w \\in R^{L−1}$\" - What is $L$? I thought $L$ is the seq length (ah yes, some paragraphs later, you define it; that should be defined when you first use it...)? But then $L$ is different each time? How can you have an infinite number of learnable weights?\n\nEq 2 / eq 3: I don't understand this. The softmax is over the $1:{L}-1$? So the $\\mathcal{W}$ shape is $[L-1]$? But then in eq 3, you multiply this with $e_k$, which is of shape $[d_{node}]$? This does not make sense.\n\nWhat shape is $E_{u,v}$?\n\nSo I assume now that $L$ is also fixed?\n\nSec 4: \"4000 ∗ 4000 ∗ (32 ∗ 32 + 32)\", don't use \"∗\" for multiplication, use `\\cdot` ($\\cdot$).\n\nSec 4: So this is enwik8 for English? Please make this explicit. Or if this is some other dataset, specify (with reference) what you use. If you use some custom dataset: Don't, unless you have a very good reason for it. But you don't. So use some publicly available dataset that your performance can directly be compared to other results from the literature. Also, 100M is considered as small nowadays. Use larger datasets.\n\nSec 4 sparse training: \"We set the connection weights corresponding to low-frequency bigrams (those not appearing in the training set) to be shared\" - please specify this more exactly. I assume I know what you do, but it would be helpful to write down (in math formulas) what you mean. Also, I don't exactly understand why you call this \"sparse\"? Sparse usually means that most of the weights are 0. But then, \"... and update them randomly\" - I don't know what you do there exactly. How do you update them randomly?\n\nTable 3: The examples look very bad. For comparison, see this article (https://karpathy.github.io/2015/05/21/rnn-effectiveness/). This is a small RNN from 2015. All the examples there look much better.\n\nReferences / related work: Please look at other papers (other NeurIPS papers), how they are written. You must put your work into context, compare it to other work. It is quite normal that you have 10s, if not 100s of references. You also completely miss a section where you compare to related work.\n\nExperiments: There are no benchmarks, where you measure perplexity, or performance on some other tasks? This is the very least that should be done. A statement like \"language model prediction performance comparable to GPT-1\" is sth which you can (and must) measure. I don't really know how you could make such a statement otherwise?\n\nLIMITATIONS:\n.", "TITLE": "Official Review", "RECOMMENDATION": 1, "REVIEWER_CONFIDENCE": 5, "CLARITY": 1, "IMPACT": 1, "ORIGINALITY": 2, "DATE": "2025-07-02"}