## 2 Transformer Symbolic Regression
### PhySO  
Thanks to [PERSON] et al., they proposed PhySO [11], a powerful deep reinforcement learning-based symbolic regression method. It builds a mathematical expression starting from the most basic physical units and automatically detects and corrects combinations of symbols that may lead to violations of physical unit constraints, ensuring that unit correctness is maintained throughout all computations. This avoids generating irrational mathematical expressions and greatly reduces the size of the expression search space, so the search process of PhySO is more efficient and can find the best solution faster.  
In the implementation, PhySO models mathematical expressions as binary trees, where variables and coefficients are terminal nodes, operation symbols are non-terminal nodes, monadic operators have only 1 child node, and binocular operators have 2 child nodes. Variables and operators are called tokens in PhySO, and the space they make up is called Library, for example, Library = {a, b, c, +, -, /}. Tokens in the Library such as \"a, b, c, +, -, /\" are encoded by one-hot for subsequent processing by the neural network.  
As a deep reinforcement learning method, PhySO sets the observations as parent nodes and their units, sibling nodes and their units, previous node and its unit, dangling nodes and the unit of the current node, and the initial observations are all zero tensor. The action is to select the tokens in the Library to build the mathematical expression. By prioritizing the output of the action by the neural network and performing the mask operation, it masks out unnecessary actions to generate better mathematical expressions. The reward is:  
\[R=\frac{1}{1+NRMSE} \tag{2}\]  
\[\sigma_{y}=\sqrt{\frac{1}{N-\delta_{N}}{\sum_{i=1}^{N}{\left({y_{i}-\frac{1}{ N}{\sum_{i=1}^{N}{y_{i}}}}\right)}^{2}}} \tag{3}\]  
\[NRMSE=\frac{1}{\sigma_{y}}\sqrt{\frac{1}{N}{\sum_{i=1}^{N}{\left({y_{i}-f(x_{ i})}\right)}^{2}}} \tag{4}\]  
Figure 3: Binary tree of formula.  
where \(R\) is the reward, \(\sigma_{y}\) is the standard deviation of the target value, \(N\) is the number of sampling points, \(\delta_{N}\) is [PERSON]'s correction and \(\delta_{N}=1\), \(x_{i},y_{i}\) are data points and target values, and \(f(\cdot)\) is the function generated by the neural network.  
The optimization strategy used by PhySO is risk-seeking policy gradient [14], entropy regularization [15], and Adam optimizer is adopted. When the reward is big enough and the expression is meaningful, the algorithm will stop iterating and output the corresponding result.  
The network architecture used by PhySO is Long Short-Term Memory (LSTM) which is a classical neural network architecture for processing sequence data. After the observations data is input into the LSTM network, the network will output the probability distribution of each action, adjust the probability distribution of the action by masking the actions that do not meet the constraint conditions, and then sample the action according to the probability. Then, based on the action, the new mathematical expression is obtained, and the observations are updated. It will be repeated until the final mathematical expression is obtained.  
By deep reinforcement learning technology, the PhySO method can adaptively adjust learning strategies, and automatically select the most appropriate mathematical expression to describe data by using physical constraints and avoid meaningless symbol combinations. This method can greatly improve the search efficiency and reliability of the model, and can better adapt to different types of physical data.