GESR: A GEOMETRIC EVOLUTION MODEL FOR SYM- BOLIC REGRESSION
GESR：一种符号回归的几何演化模型

Figure 1: Schematic of the proposed method. For each symbolic tree in the population, the GESR first selects the mutated subtree from a symbolic tree based on the calculated probabilities. The chosen subtree is then replaced by the generated combined tree through the semantics approximation in the topological space. Periodically, each tree in the population will be adjusted by the Weight Optimization module. The generated offspring is then executed to get the fitness value and the assisted information needed in the next iteration. After the execution, the evaluation and selection are performed and the selected symbolic trees form the new population for the next round iteration. These procedures are detailedly described in Section 3
图 1：所提方法的示意图。对于种群中的每个符号树，GESR 首先根据计算出的概率从符号树中选择待变异的子树。接着，通过拓扑空间中的语义近似，将选中的子树替换为生成的组合树。周期性地，种群中的每棵树会通过权重优化模块进行调整。生成的子代随后被执行以获取适应度值及下一次迭代所需的辅助信息。执行完毕后进行评估与选择，被选中的符号树构成新一轮迭代的新种群。这些步骤在第 3 节中有详细描述。

high-dimensional nature of semantic space, these methods still struggle in the complex intermediate processes to reach the target semantics. Moreover, the direct mapping of geometric operations to syntax modifications also leads to severe tree bloating issues. Searching for target semantics in high-dimensional semantic space is still a challenging task.
尽管语义空间具有高维特性，这些方法在抵达目标语义的复杂中间过程中仍面临困难。此外，将几何操作直接映射到语法修改也会导致严重的树膨胀问题。在高维语义空间中寻找目标语义仍是一项具有挑战性的任务。

Given a dataset $\mathcal{D}={\left\{({\mathbf{x}}_i,{\mathbf{y}}_i)\right\}}_{i<N},\left({\mathbf{x}}_i,{\mathbf{y}}_i\right)\in {\mathbb{R}}^d\times \mathbb{R}$ ,the main goal of symbolic regression (SR) methods is to find a symbolic expression $f^{\ast }$ that minimizes the theoretical risk of the input-output mapping of the dataset $\mathcal{D}$ :
给定一个数据集 $\mathcal{D}={\left\{({\mathbf{x}}_i,{\mathbf{y}}_i)\right\}}_{i<N},\left({\mathbf{x}}_i,{\mathbf{y}}_i\right)\in {\mathbb{R}}^d\times \mathbb{R}$ ，符号回归（SR）方法的主要目标是找到一个符号表达式 $f^{\ast }$ ，以最小化数据集 $\mathcal{D}$ 输入输出映射的理论风险：

Semantic genetic programming (SGP) vectorizes the output of the symbolic expression related to the dataset $\mathcal{D}$ as a semantic vector,with which the iterative improvement of the symbolic expression can be achieved through semantic target approximation in topological space. The semantic vectors of possible solutions constitute semantic space, where each symbolic expression is mapped to a semantic vector point. In this paper, we propose a geometric semantic genetic programming algorithm, which utilizes geometric properties describing spatial relationships between symbolic expressions in the $\mathrm{n}$ -dimensional semantic space,where $\mathrm{n}$ is equivalent to the size of the training set. By directly approximating the target semantic with a geometric method in the semantic space and mapping the corresponding geometric operations to the syntactic modifications of symbolic expression, newly
语义遗传编程（SGP）将与数据集 $\mathcal{D}$ 相关的符号表达式输出向量化为语义向量，通过语义目标在拓扑空间中的逼近实现符号表达式的迭代优化。可能解的语义向量构成语义空间，其中每个符号表达式映射为一个语义向量点。本文提出一种几何语义遗传编程算法，利用描述 $\mathrm{n}$ 维语义空间中符号表达式间空间关系的几何特性（其中 $\mathrm{n}$ 等同于训练集大小），通过在语义空间中直接以几何方法逼近目标语义，并将对应的几何操作映射到符号表达式的语法修改上，新生成的...

obtained symbolic expression can be closer to the target. For example, the midpoint of the two semantic points ${\mathbf{s}}^1$ and ${\mathbf{s}}^2$ that represent two symbolic expression trees $t{r}^1$ and $t{r}^2$ respectively in the semantic space can be mapped to a new expression tree: ${\mathbf{s}}^{\prime }:1/2{\mathbf{s}}^1+1/2{\mathbf{s}}^2\to t{r}^{\prime }$ : $1/2\ast t{r}^1+1/2\ast t{r}^2$ .
获得的符号表达式可以更接近目标。例如，语义空间中分别代表两个符号表达式树 $t{r}^1$ 和 $t{r}^2$ 的两个语义点 ${\mathbf{s}}^1$ 和 ${\mathbf{s}}^2$ 的中点可以映射到一个新的表达式树： ${\mathbf{s}}^{\prime }:1/2{\mathbf{s}}^1+1/2{\mathbf{s}}^2\to t{r}^{\prime }$ : $1/2\ast t{r}^1+1/2\ast t{r}^2$ 。

In our method, geometric semantics is used to guide the mutation, as shown in Figure 1 However, although geometric semantic mutation improves search efficiency and accuracy, frequent linear combinations of expressions often lead to severe tree bloating. Therefore, we backpropagate the target semantics to sub-expression, combined with semantic gradients and strict limitations, to guide the mutation process in the sub-semantic space. The linear combination of sub-expressions also leads to a significant increase in the proportion of constants. While geometric semantic mutation based on backpropagation is restricted to only adjusting local constants, it is difficult to balance the optimization of global constants. Thus, we incorporate the Levenberg-Marquardt optimization algorithm with L2 regularization to periodically adjust global constants, which also enables the smoothing of formula curves. By the way, since we assign a constant (weight) to each combination subtree, there is a natural characteristic for our method to use a continuous optimization algorithm to further adjust the overall tree structures.
本方法采用几何语义来指导变异过程，如图 1 所示。然而，尽管几何语义变异提升了搜索效率与精度，但频繁的表达式线性组合常导致严重的树结构膨胀。为此，我们将目标语义反向传播至子表达式，结合语义梯度与严格限制，在子语义空间内引导变异过程。子表达式的线性组合还会导致常数占比显著上升。虽然基于反向传播的几何语义变异仅限调整局部常数，但难以兼顾全局常数的优化。因此，我们引入带 L2 正则化的 Levenberg-Marquardt 优化算法周期性调整全局常数，同时实现公式曲线的平滑化。值得一提的是，由于我们为每个组合子树分配了常数（权重），该方法天然具备使用连续优化算法进一步调整整体树结构的特性。

Semantic backpropagation is achieved by inversely calculating the expression tree (Krawiec & Pawlak, 2013). As the target semantics are reversed into sub-target semantics, the semantic space is also mapped to a sub-semantic space. Thus the solution in the semantic space can be found by searching within the sub-semantic space. In this section, a semantic gradient concept is further proposed to assist the exploration in semantic space, which comes from the following observation: The approximation process of the sub-semantic space cannot be equivalently propagated to the root semantic space. Figure 2 is an example for better understanding. While the target semantics can be backpropagated to the sub-semantic space, the approximation process will be seriously affected by the calculation path of the mutation node, where the calculation path serves as the mapping function $f\left(y\right)$ . With $f\left(y\right)$ ,the evaluation based on the Euclidean distance between the semantics $\mathbf{s}$ of subtree $tr$ and the sub-target semantics st in the sub-semantic space: $\sqrt{\sum _{i=0}^n{({\mathbf{s}}_i-{\mathbf{st}}_i)}^2}$ ,is transformed into $\sqrt{\sum _{i=0}^n{(f\left({\mathbf{s}}_i\right)-f\left({\mathbf{st}}_i\right))}^2}\Rightarrow \sqrt{\sum _{i=0}^n{(w\cdot 2{\mathbf{x}}_i)}^2\cdot {({\mathbf{s}}_i-{\mathbf{st}}_i)}^2}$ . The uncertain vector $\mathbf{x}$ makes each dimension of the approximation result from the sub-semantic space deviate in the target semantic space, which significantly reduces the accuracy of semantic approximation based on backpropagation. To address this issue, we introduce the concept of semantic gradients to approximate the mapping
语义反向传播通过逆向计算表达式树实现（Krawiec & Pawlak, 2013）。当目标语义被反向分解为子目标语义时，语义空间也随之映射到子语义空间。因此，通过在子语义空间内搜索即可找到语义空间中的解。本节进一步提出语义梯度概念以辅助语义空间探索，该概念源于以下观察：子语义空间的逼近过程无法等效传播至根语义空间。图 2 为例便于理解：虽然目标语义可反向传播至子语义空间，但逼近过程会显著受变异节点计算路径影响，其中计算路径充当映射函数 $f\left(y\right)$ 。借助 $f\left(y\right)$ ，基于子树 $tr$ 的语义 $\mathbf{s}$ 与子语义空间中子目标语义 st 的欧氏距离评估： $\sqrt{\sum _{i=0}^n{({\mathbf{s}}_i-{\mathbf{st}}_i)}^2}$ ，被转化为 $\sqrt{\sum _{i=0}^n{(f\left({\mathbf{s}}_i\right)-f\left({\mathbf{st}}_i\right))}^2}\Rightarrow \sqrt{\sum _{i=0}^n{(w\cdot 2{\mathbf{x}}_i)}^2\cdot {({\mathbf{s}}_i-{\mathbf{st}}_i)}^2}$ 。 不确定向量 $\mathbf{x}$ 使得子语义空间中的近似结果在目标语义空间的每个维度上发生偏离，这显著降低了基于反向传播的语义近似精度。为解决这一问题，我们引入语义梯度的概念来近似映射

Figure 2: Spatial transformation under nonlinear mapping function $\mathrm{f}\left(\mathrm{y}\right)$ . With $\mathrm{f}\left(\mathrm{y}\right)$ ,the Euclidean distance between semantic points in the sub-semantic space changes when mapped to the target semantic space.
图 2：非线性映射函数 $\mathrm{f}\left(\mathrm{y}\right)$ 下的空间变换。当使用 $\mathrm{f}\left(\mathrm{y}\right)$ 时，子语义空间中语义点之间的欧几里得距离在映射到目标语义空间时会发生变化。

relationship between the sub-semantic space and the target semantic space. As illustrated in Figure 3 starting from the root node of the expression tree, the semantic gradients of the mutated node are computed along the semantic backpropagation path,thereby obtaining the gradient vector $\mathrm{\nabla }T$ for the
子语义空间与目标语义空间之间的关系。如图 3 所示，从表达式树的根节点出发，沿着语义反向传播路径计算变异节点的语义梯度，从而得到梯度向量 $\mathrm{\nabla }T$ 。

216

Figure 3: The process of semantic gradient computation. The semantic vectors ${\mathbf{s}}^i$ on the backpropa-gation path are replaced by the corresponding sub-target semantic ${\mathbf{st}}_i$ .
图 3：语义梯度计算过程。反向传播路径上的语义向量 ${\mathbf{s}}^i$ 被相应的子目标语义 ${\mathbf{st}}_i$ 所替代。

218

$j^{th}$ node. The representation of $\mathrm{\nabla }T$ is as follows:
$j^{th}$ 节点。 $\mathrm{\nabla }T$ 的表示如下：

Where ${\mathbf{s}}_i^j$ denotes the $i^{th}$ dimension of the semantics of the $j^{th}$ subtree. The partial derivative values across various semantic dimensions indicate the changing rate of the nonlinear mapping function $f\left(t{r}^j\right)$ with respect to the semantics of the subtree $t{r}^j$ . Specifically,the absolute partial derivative values across each semantic dimension serve as an indicator of the degree of influence that deviations of $t{r}^j$ within the corresponding sub-semantic space have on the deviations of $f\left(t{r}^j\right)$ within the corresponding dimension of the target semantic space. Thus,the normalized value ${\left|\mathrm{\nabla }T\right|}_{\mathcal{N}}^j$ serves as the weight of each dimension in the sub-semantic space. As the gradient vector is affected by the semantics of the mutated node which is unfixed, we use the sub-target semantics st instead of the current mutated node semantics $\mathbf{s}$ when computing the semantic gradient:
其中 ${\mathbf{s}}_i^j$ 表示 $j^{th}$ 子树语义的第 $i^{th}$ 个维度。各语义维度上的偏导数值反映了非线性映射函数 $f\left(t{r}^j\right)$ 相对于子树语义 $t{r}^j$ 的变化率。具体而言，每个语义维度上偏导数的绝对值大小，反映了对应子语义空间中 $t{r}^j$ 的偏差对目标语义空间相应维度上 $f\left(t{r}^j\right)$ 偏差的影响程度。因此，归一化后的值 ${\left|\mathrm{\nabla }T\right|}_{\mathcal{N}}^j$ 作为子语义空间各维度的权重。由于梯度向量受未固定的变异节点语义影响，在计算语义梯度时，我们采用子目标语义 st 而非当前变异节点语义 $\mathbf{s}$ 。

Where $\mathbf{s}{t}_i^j$ refers to the sub-target semantics of the $i^{th}$ dimension of the $j^{th}$ subtree.
其中 $\mathbf{s}{t}_i^j$ 指代 $j^{th}$ 子树第 $i^{th}$ 维度的子目标语义。

Due to the high-dimensional nature of the semantic space, in which the dimensions equal to the training set size, the sparsity of semantic points resulting from high-dimensional features makes it difficult for geometric semantic mutation strategies to search for target semantics in the semantic space. To quickly approximate the target semantics in the semantic space, two steps are conducted as follows. First, as the distribution of candidates in the semantic space is sparse, we scale each candidate semantics $s$ to the perpendicular projection of the sub-target semantics onto the line connecting the semantics point and the origin point. This step aims to migrate the distribution of candidate semantics s to the sub-target semantics st around the semantic space, as shown below:
由于语义空间的高维特性（其维度等于训练集大小），高维特征导致的语义点稀疏性使得几何语义突变策略难以在语义空间中搜索目标语义。为了快速逼近语义空间中的目标语义，我们采取以下两个步骤：首先，鉴于语义空间中候选点的分布稀疏，我们将每个候选语义 $s$ 缩放至子目标语义在该语义点与原点连线上的垂直投影位置。此步骤旨在将候选语义 s 的分布迁移至子目标语义 st 周围，如下所示：

where, $\alpha$ :  其中， $\alpha$ ：

Subsequently,we adopt $\mathrm{var}\left(\mathbf{st}-\alpha \cdot \mathbf{s}\right)=\sum _{i=1}^n{(({\mathbf{st}}_i-\alpha \cdot {\mathbf{s}}_i)-E({\mathbf{st}}_i-\alpha \cdot {\mathbf{s}}_i))}^2$ to rank the candidate set, instead of utilizing the Euclidean distance metric, aiming to get the subexpressions that are more similar in functional form rather than a smaller scalar distance. The possible constant deviation can be eliminated with the constant node in the candidate set, as detailed in Appendix B The approximated candidate set then is sorted based on the Euclidean distance $\mathrm{dis}\left({\mathbf{s}}_i^{\prime },\mathbf{st}\right)$ between the semantics of candidates and the sub-target semantics. After that, we select multiple pair of
随后，我们采用 $\mathrm{var}\left(\mathbf{st}-\alpha \cdot \mathbf{s}\right)=\sum _{i=1}^n{(({\mathbf{st}}_i-\alpha \cdot {\mathbf{s}}_i)-E({\mathbf{st}}_i-\alpha \cdot {\mathbf{s}}_i))}^2$ 对候选集进行排序，而非使用欧氏距离度量，旨在获取函数形式更为相似而非标量距离更小的子表达式。如附录 B 所述，候选集中的常数节点可消除可能的常量偏差。接着，基于候选者语义与子目标语义之间的欧氏距离 $\mathrm{dis}\left({\mathbf{s}}_i^{\prime },\mathbf{st}\right)$ 对近似候选集进行排序。此后，我们选取多组

candidates in order and use the least squares method to combine them pairwise to find the linear combination of candidates that best approximates the target semantics. For the selection of $t{r}^1$ ,we select the $top-t$ subtrees within the sorted set as candidates. For the selection of $t{r}^2$ ,the following $m$ subtrees for each $t{r}^1$ are chosen to form the candidate set. Then,the obtained candidate subtrees from $t{r}^1$ set and $t{r}^2$ set are linearly combined to minimize the distance $\mathcal{L}\left(\mathbf{st},f\left({\mathbf{s}}^{tr1},{\mathbf{s}}^{tr2}\right)\right)$ ,which can be represented as below with Eq 4:
候选者并按顺序使用最小二乘法将它们两两组合，以找到能最佳逼近目标语义的候选者线性组合。关于 $t{r}^1$ 的选择，我们从排序集中选取 $top-t$ 棵子树作为候选者。对于 $t{r}^2$ 的选择，则为每个 $t{r}^1$ 选取后续 $m$ 棵子树构成候选集。随后，将来自 $t{r}^1$ 集和 $t{r}^2$ 集的候选子树进行线性组合以最小化距离 $\mathcal{L}\left(\mathbf{st},f\left({\mathbf{s}}^{tr1},{\mathbf{s}}^{tr2}\right)\right)$ ，其数学表示如式 4 所示：

Here,by utilizing the least squares method to minimize the loss function $\mathcal{L}\left(\mathbf{st},f\left({\mathbf{s}}^{tr1},{\mathbf{s}}^{tr2}\right)\right)=$ $\sum {(\mathbf{s}{\mathbf{t}}_i-f({\mathbf{s}}_i^{tr1},{\mathbf{s}}_i^{tr2}))}^2\cdot {\left|\mathrm{\nabla }T\right|}_{\mathcal{N},i}=\sum {(\mathbf{s}{\mathbf{t}}_i-{\alpha }_1\cdot (1-k)\cdot {\mathbf{s}}_i^{tr1}-k\cdot {\alpha }_2\cdot {\mathbf{s}}_i^{tr1})}^2\cdot {\left|\mathrm{\nabla }T\right|}_{\mathcal{N},i},$ we can obtain $k$ as follows:
此处，通过最小二乘法最小化损失函数 $\mathcal{L}\left(\mathbf{st},f\left({\mathbf{s}}^{tr1},{\mathbf{s}}^{tr2}\right)\right)=$ $\sum {(\mathbf{s}{\mathbf{t}}_i-f({\mathbf{s}}_i^{tr1},{\mathbf{s}}_i^{tr2}))}^2\cdot {\left|\mathrm{\nabla }T\right|}_{\mathcal{N},i}=\sum {(\mathbf{s}{\mathbf{t}}_i-{\alpha }_1\cdot (1-k)\cdot {\mathbf{s}}_i^{tr1}-k\cdot {\alpha }_2\cdot {\mathbf{s}}_i^{tr1})}^2\cdot {\left|\mathrm{\nabla }T\right|}_{\mathcal{N},i},$ ，我们可得到 $k$ 如下：

In the Eq ${|\mathcal{T},{\left|\mathrm{\nabla }T\right|}_{\mathcal{N}}|}_{\mathcal{N}}$ represents the normalized vector of the absolute semantic gradient. With Eq 4. 5 6.7,a new combined tree can be generated: $t{r}^{\prime }={\alpha }_1\cdot \left(1-k\right)\cdot t{r}^1+{\alpha }_2\cdot k\cdot t{r}^2$ . For the generated candidates $\left\{t{r}^{\prime 1},t{r}^{\prime 2},\dots ,t{r}^{\prime n}\right\}$ after pairwise linear combinations,we first filter the candidate set based on the Euclidean distance $\mathcal{L}\left(\mathbf{st},{\mathbf{s}}^{\prime }\right)$ between the semantics ${\mathbf{s}}^{\prime }$ of each combined candidate subtree $t{r}^{\prime }$ and the sub-target semantics st in the root semantic space. Considering that the Euclidean distance used as the evaluation criterion is a scalar value, which loses the vector information in the semantic space. Simply pursuing a decrease in distance may lead to a loss of diversity and trap into local optima. Therefore,an additional parameter $\lambda$ is used to fully explore the solution space:
在方程 ${|\mathcal{T},{\left|\mathrm{\nabla }T\right|}_{\mathcal{N}}|}_{\mathcal{N}}$ 中代表绝对语义梯度的归一化向量。通过方程 4.5、6.7，可以生成一个新的组合树： $t{r}^{\prime }={\alpha }_1\cdot \left(1-k\right)\cdot t{r}^1+{\alpha }_2\cdot k\cdot t{r}^2$ 。对于经过两两线性组合生成的候选集 $\left\{t{r}^{\prime 1},t{r}^{\prime 2},\dots ,t{r}^{\prime n}\right\}$ ，我们首先基于欧氏距离 $\mathcal{L}\left(\mathbf{st},{\mathbf{s}}^{\prime }\right)$ 对候选集进行筛选，该距离衡量的是每个组合候选子树 $t{r}^{\prime }$ 的语义 ${\mathbf{s}}^{\prime }$ 与根语义空间中子目标语义 st 之间的差异。考虑到作为评估标准的欧氏距离是一个标量值，会丢失语义空间中的向量信息，单纯追求距离减小可能导致多样性丧失并陷入局部最优。因此，引入额外参数 $\lambda$ 以充分探索解空间：

Where ${\mathbf{s}}^c$ refers to the semantics of the original subtree,and the candidates that do not satisfy the Eq 8 are filtered out. Moreover, we incorporate a variance-based constraint method to ensure smoother function fitting, thereby avoiding potential issues of local overfitting caused by uneven data distribution or significant differences of output values within the dataset. The constraint is formulated
其中 ${\mathbf{s}}^c$ 表示原子树的语义，不满足方程 8 的候选将被过滤掉。此外，我们采用基于方差的约束方法以确保更平滑的函数拟合，从而避免因数据分布不均或数据集内输出值差异显著可能引发的局部过拟合问题。该约束条件表述

as follows:  如下：

The remaining candidate set that satisfies both Eq. 8 and Eq. 9 is evaluated in the final step. Since the semantics is solely constituted by output vectors, lacking tree structure information, which results in a severe expansion problem when translating the semantic space approximation to expression trees. Inspired by the SPL approach (Sun et al. 2022), the final evaluation function is calculated using the following formula:
在最终步骤中，对同时满足式 8 和式 9 的剩余候选集进行评估。由于语义仅由输出向量构成，缺乏树结构信息，这导致在将语义空间近似转换为表达式树时出现严重的扩展问题。受 SPL 方法（Sun 等人，2022）启发，最终评估函数采用以下公式计算：

Where $\eta$ is the discount factor promoting concise trees, $l$ represents the mutated subtree size. We only replace the mutated subtrees where $\mathcal{R}$ is smaller than the original ${\mathcal{R}}^c$ . It is worth noting that the mutated subtree size is calculated by the inner node size instead of the tree size to ensure the fairness of each function symbol. By employing this evaluation function, our method encourages finding more concise expressions while minimizing the Euclidean distance.
其中 $\eta$ 是鼓励简洁树的折扣因子， $l$ 代表变异子树的大小。我们仅替换那些 $\mathcal{R}$ 小于原 ${\mathcal{R}}^c$ 的变异子树。值得注意的是，变异子树的大小是通过内部节点大小而非整树大小计算的，以确保每个函数符号的公平性。通过采用这一评估函数，我们的方法在最小化欧氏距离的同时，鼓励寻找更简洁的表达式。

In selecting a subtree in a symbolic tree to mutate, the traditional random selection strategy without any prior information reduces search efficiency. We integrate semantic information with the semantic gradient into the mutated subtree selection strategy to guide the selection of mutated subtrees, where the Euclidean distance is chosen as the basic metric for assessing the optimization potential of candidate mutated subtrees. By computing the Euclidean distance between the semantics of the candidate mutated subtree and the sub-target semantics, and mapping it through the semantic gradient to the root semantic space, we obtain a scalar expression of the potential for each candidate mutated subtree. Then, we utilize softmax to convert the mapped Euclidean distance of each candidate mutated
在符号树中选择子树进行变异时，传统无先验信息的随机选择策略会降低搜索效率。我们将语义信息与语义梯度相结合，融入变异子树选择策略以指导变异子树的选取，其中选用欧几里得距离作为评估候选变异子树优化潜力的基础度量标准。通过计算候选变异子树语义与子目标语义间的欧氏距离，并经由语义梯度映射至根语义空间，我们得到每个候选变异子树潜力的标量化表达。随后，利用 softmax 将各候选变异子树的映射欧氏距离转化为概率，并采用锦标赛选择策略从候选子树中确定最终的变异子树。

subtree into probabilities and use a tournament selection strategy to choose the final mutated subtree from the candidate subtrees. The selected probability $p\left(t{r}^i\right)$ of each candidate $t{r}^i$ is computed by:
子树转化为概率后，通过锦标赛选择策略从候选子树中选出最终的变异子树。每个候选子树 $t{r}^i$ 的被选概率 $p\left(t{r}^i\right)$ 计算公式如下：

Where ${\mathbf{r}}^i={({\mathbf{st}}^i-{\mathbf{s}}^i)}^2\cdot {\left|\mathrm{\nabla }T\right|}_{\mathcal{N}}^i$ and $M=\underset{0\le i\le n}{max}{\mathbf{r}}^i=\underset{0\le i\le n}{max}\left({({\mathbf{st}}^i-{\mathbf{s}}^i)}^2\cdot {\left|\mathrm{\nabla }T\right|}_{\mathcal{N}}^i\right)$ is used to prevent data overflow. It is worth noting that we only randomly select a few subtrees as candidates in the expression tree in each generation to make full exploration while ensuring randomness, since the mismatched parts of an expression may be scattered across multiple branches of the expression tree.
其中 ${\mathbf{r}}^i={({\mathbf{st}}^i-{\mathbf{s}}^i)}^2\cdot {\left|\mathrm{\nabla }T\right|}_{\mathcal{N}}^i$ 和 $M=\underset{0\le i\le n}{max}{\mathbf{r}}^i=\underset{0\le i\le n}{max}\left({({\mathbf{st}}^i-{\mathbf{s}}^i)}^2\cdot {\left|\mathrm{\nabla }T\right|}_{\mathcal{N}}^i\right)$ 用于防止数据溢出。值得注意的是，在每一代生成过程中，我们仅从表达式树中随机选取少数子树作为候选，以确保在充分探索的同时维持随机性，因为表达式的不匹配部分可能分散在表达式树的多个分支中。

The optimization of geometric semantic mutation lies solely in the adjustment of local tree structures, and it is worth noting that our geometric semantics method provides weight coefficients for each subtree(Equation 6). Differs from the traditional methods that use a continuous optimization algorithm to optimize the constants (Kommenda et al., 2020; De Melo et al., 2015), the presence of these weight coefficients allows us not only to adjust each subtree module at any time but also, more importantly, to automatically adjust the overall structure by optimizing constants and removing unsuitable subtrees (by setting the corresponding subtree's weight coefficient to 0.0 , when the weight coefficient is approximated to nearly zero(<1e-6 in our method)). Therefore, we employ the Levenberg-Marquardt(LM) (Moré, 2006) algorithm to perform constant optimization on the expressions at regular intervals. The LM algorithm utilizes Euclidean distance as the loss function. Additionally, considering that expressions generated based on geometric semantics often contain numerous constants, we have added L2 regularization to smooth the function curve, as shown below:
几何语义突变的优化仅在于局部树结构的调整，值得注意的是，我们的几何语义方法为每个子树提供了权重系数（方程 6）。与传统方法使用连续优化算法优化常量（Kommenda 等人，2020；De Melo 等人，2015）不同，这些权重系数的存在使我们不仅能够随时调整每个子树模块，更重要的是，通过优化常量并移除不合适的子树（当权重系数接近零时（在我们的方法中<1e-6），将相应子树的权重系数设置为 0.0），自动调整整体结构。因此，我们采用 Levenberg-Marquardt（LM）（Moré，2006）算法定期对表达式进行常量优化。LM 算法利用欧几里得距离作为损失函数。此外，考虑到基于几何语义生成的表达式通常包含大量常量，我们添加了 L2 正则化以平滑函数曲线，如下所示：

Following with LM algorithm,we can optimize the constants $\mathbf{w}$ through the formula:
遵循 LM 算法，我们可以通过公式优化常量 $\mathbf{w}$ ：

Where ${\mathbf{J}}_r$ is the Jacobian matrix of constants, $\mathbf{I}$ is the identity matrix, $\mu$ is the penalty factor, $\beta$ is the hyperparameter,and $\mathbf{w}$ represents constants within the expression.
其中 ${\mathbf{J}}_r$ 为常数雅可比矩阵， $\mathbf{I}$ 为单位矩阵， $\mu$ 为惩罚因子， $\beta$ 为超参数， $\mathbf{w}$ 代表表达式中的常数项。

We assess GESR on two mainstream large-scale datasets: the SRBench benchmark (La Cava et al. 2021) which includes both real-world and ground-truth problems, and the SRSD benchmark (Matsub-ara et al. 2022) for scientific discovery. A total of 25 baseline methods are used for performance comparison in the experiment. It is worth noting that these baseline methods use multiple parameter sets and find the most suitable parameter settings through a half-grid search. However, considering the significant impact of parameters on algorithm performance, to demonstrate the superiority of our method and reduce the influence of parameter selection, the hyper-parameters of our method remain the same on all datasets to control the influencing factors. The detailed hyper-parameter setting and the brief introductions to the baseline methods are provided in Appendix A and C respectively. The PMLB dataset includes 46 real-world black-box datasets, including physics, home price, etc., 76 synthetic datasets including Friedman datasets, 130 Strogatz datasets, and Feynman datasets. Among them, real-world datasets and Friedman datasets(form the black-box datasets) are used for overall evaluations, Friedman datasets are used for ablation analysis, Strogatz and Feynman datasets with different levels of Gaussian noise are used for robustness verification. In addition, the SRSD benchmark with 120 scientific discovery datasets that contain dummy variables is also used to test the effectiveness of GESR, in which the properties of the formula and the variables are carefully reviewed to ensure a realistic sampling value range. Our method is assessed with a popular metrics: $R^2$ -score (La Cava et al. 2021).
我们在两个主流大规模数据集上评估 GESR：包含现实世界和基准真实问题的 SRBench 基准测试（La Cava 等人，2021 年），以及用于科学发现的 SRSD 基准测试（Matsubara 等人，2022 年）。实验中总共使用了 25 种基线方法进行性能比较。值得注意的是，这些基线方法采用多组参数，并通过半网格搜索找到最合适的参数设置。然而，考虑到参数对算法性能的重大影响，为了展示我们方法的优越性并减少参数选择的影响，我们方法在所有数据集上的超参数保持一致以控制影响因素。详细的超参数设置和基线方法的简要介绍分别见附录 A 和 C。PMLB 数据集包含 46 个现实世界黑盒数据集（涉及物理、房价等领域）、76 个合成数据集（包括 Friedman 数据集）、130 个 Strogatz 数据集以及 Feynman 数据集。 其中，真实世界数据集和弗里德曼数据集（构成黑盒数据集）用于整体评估，弗里德曼数据集用于消融分析，斯托加茨和费曼数据集则添加不同级别的高斯噪声以进行鲁棒性验证。此外，还采用了包含虚拟变量的 120 个科学发现数据集组成的 SRSD 基准来测试 GESR 的有效性，这些数据集中公式和变量的属性经过仔细审查，以确保采样值范围的合理性。我们的方法采用了一项流行指标进行评估： $R^2$ 分数（La Cava 等人，2021 年）。

378

Table 1: SRSD+ Dummy Variables: Accuracy solution rate $\left(R^2>0.999\right)$ on scientific discovery datasets with different complexity.
表 1：SRSD+虚拟变量：不同复杂度科学发现数据集上的准确解决率 $\left(R^2>0.999\right)$ 。

380

We evaluated our method on the SRSD benchmark that contains 120 scientific discovery datasets with dummy variables, to assess the performance in tackling scientific discovery problems. As shown in Table 1,our method achieved accuracy solution rates $\left(R^2>0.999\right)$ of $100.0\mathrm{\%},87.5\mathrm{\%}$ ,and $58\mathrm{\%}$ on the easy, medium, and hard datasets, which means that our method gets the improvements of 23.3%, $42.5\mathrm{\%},36\mathrm{\%}$ over the second-ranked baseline method respectively. The experimental result shows the powerful search ability of our geometric semantic method in approximating target semantics, especially in addressing complex scientific discovery problems.The further experimental results on the SRSD benchmark are presented in Appendix D.3, and the qualitative analysis of the GESR has been discussed in Appendix D.4
我们在包含 120 个带有虚拟变量的科学发现数据集的 SRSD 基准上评估了我们的方法，以评估其在解决科学发现问题中的性能。如表 1 所示，我们的方法在简单、中等和困难数据集上分别实现了准确解决率 $\left(R^2>0.999\right)$ 、 $100.0\mathrm{\%},87.5\mathrm{\%}$ 和 $58\mathrm{\%}$ ，这意味着我们的方法相对于排名第二的基线方法分别提高了 23.3%和 $42.5\mathrm{\%},36\mathrm{\%}$ 。实验结果表明，我们的几何语义方法在逼近目标语义方面具有强大的搜索能力，特别是在解决复杂的科学发现问题时。关于 SRSD 基准的进一步实验结果见附录 D.3，GESR 的定性分析已在附录 D.4 中讨论。

400

Figure 4: SRBench-generated comparisons of $R^2$ test(left) and model size(right) on 120 black-box datasets.
图 4：SRBench 生成的关于 120 个黑盒数据集的 $R^2$ 测试（左）和模型大小（右）的比较。

403

408

Figure 4 presents the performance of GESR on the SRBench benchmark. In terms of overall accuracy, our method outperforms all other algorithms. Additionally, our method also exhibits advantages in model size and shows substantial improvements compared to previous semantics-based approaches (SBP-GP).
图 4 展示了 GESR 在 SRBench 基准测试中的性能表现。在整体准确率方面，我们的方法超越了所有其他算法。此外，该方法在模型规模上也展现出优势，相比此前基于语义的方法（SBP-GP）有显著提升。

In addition to conducting large-scale evaluations on the scientific discovery problems within the SRSD benchmark and real-world datasets from the SRBench benchmark, we have also tested our method on several common datasets (Nguyen, Livermore). Further experimental details have been provided in Appendix C.
除了在 SRSD 基准的科学发现问题及 SRBench 基准的真实数据集上进行大规模评估外，我们还在多个常用数据集（Nguyen、Livermore）上测试了本方法。更多实验细节详见附录 C。

In our method, symbolic expression is generated through the direct data fitting at the semantic level. This may give the impression that the GESR is weak in noise robustness ability. To further explore the adaptability of the GESR to noise and the capability to solve the scientific discovery datasets, we conducted additional experiments on the synthetic dataset Strogatz with different levels of Gaussian noise.
本方法通过在语义层面直接拟合数据生成符号表达式，这可能让人误以为 GESR 的抗噪声能力较弱。为深入探究 GESR 对噪声的适应性及解决科学发现数据集的能力，我们在合成数据集 Strogatz 上针对不同强度的高斯噪声开展了补充实验。

432

Figure 5: Accuracy solution rate $\left(R^2>0.999\right)$ on Feynman datasets(left) and Strogatz datasets(right) of SRBench benchmark with different noise levels.
图 5：SRBench 基准测试中，不同噪声水平下 Feynman 数据集（左）和 Strogatz 数据集（右）的准确解决率 $\left(R^2>0.999\right)$ 。

433

434

436

438

439

440

441

443

444

As depicted in Figure 5, our method gets state-of-the-art performance. Compared to the black-box datasets with the unknown underlying data generated function containing substantial irregular noise and dummy variables, there a more pronounced distinctions among the baseline methods for the capability to solve the scientific discovery datasets. It can be observed that the Gaussian noise has a negative impact on the accuracy solution rate of most methods. The stable accuracy solution rate of GESR than other methods for different levels of Gaussian noise demonstrates the certain degree of the noise-resistant capabilities of our method. The specific performance of our method on each dataset is provided in Appendix D.6, in which it can be observed that our method outperforms others on most datasets in terms of median $R^2$ ,mean $R^2$ ,and solution rate.
如图 5 所示，我们的方法取得了最先进的性能。与包含大量不规则噪声和虚拟变量、底层数据生成函数未知的黑盒数据集相比，科学发现数据集的基线方法在解决能力上表现出更显著的差异。可以观察到，高斯噪声对大多数方法的准确解决率产生了负面影响。GESR 在不同水平高斯噪声下稳定的准确解决率，证明了我们的方法具有一定程度的抗噪声能力。附录 D.6 提供了我们的方法在每类数据集上的具体表现，从中可见在多数数据集上，我们的方法在中位数 $R^2$ 、均值 $R^2$ 和解决率方面均优于其他方法。

In GESR, several components have been integrated. To validate the necessity of each component, we conduct ablation experiments on the Friedman dataset to individually examine the effects of the semantic gradient strategy, semantic mutation point selection strategy, geometric semantic approximation strategy, and constant optimization. Additionally, the full GESR method is used as a baseline. Table 2 shows the average $R^2$ score,average $R^2$ rank,and average model size on Friedman
在 GESR 中，我们整合了多个组件。为验证各模块的必要性，我们在 Friedman 数据集上进行了消融实验，分别考察语义梯度策略、语义突变点选择策略、几何语义近似策略以及常数优化的效果。同时以完整 GESR 方法作为基线。表 2 展示了 Friedman 数据集上的平均 $R^2$ 得分、平均 $R^2$ 排名及平均模型大小。

Table 2: The experimental results on Friedman datasets with the removal or replacement of each component. GESR-opt, GESR-slt, and GESR-gradient correspond to the removal of the constant optimization module, the semantic mutation point selection module, and the semantic gradient module. GESR-mutation refers to the replacement of the geometric semantic approximation module to linear scale strategy in (Virgolin et al. 2019). The GESR-baseline refers to the complete GESR method.
表 2：Friedman 数据集上移除或替换各组件后的实验结果。GESR-opt、GESR-slt 和 GESR-gradient 分别对应移除常数优化模块、语义突变点选择模块和语义梯度模块。GESR-mutation 指将几何语义近似模块替换为(Virgolin 等人 2019 年提出的)线性缩放策略。GESR-baseline 表示完整的 GESR 方法。

datasets with the lack or replacement of each component. Generally from the experimental results, the removal or replacement of any component shows a negative impact on our method, no matter the average $R^2$ or the average rank,which highlights the necessity of the proposed components. To further test whether there are statistically significant difference in performance with the removal of replacement of each component, a Wilcoxon signed-rank test (Wilcoxon, 1992) is performed on
数据集因各组件缺失或被替换的情况。总体实验结果表明，无论移除或替换哪个组件，都会对我们的方法产生负面影响，无论是平均 $R^2$ 还是平均排名，这凸显了所提出组件的必要性。为进一步测试移除或替换各组件后性能是否存在统计学上的显著差异，我们对中位数实验结果进行了 Wilcoxon 符号秩检验(Wilcoxon, 1992)。与 GESR 基线相比，GESR-opt、GESR-slt、GESR-gradient 和 GESR-mutation 的 p 值分别低于 1e-6、1e-2、1e-7 和 1e-6，表明在包含或不包含所提出的权重优化模块、语义点锦标赛选择模块、几何语义方法和语义梯度模块时， $R^2$ 存在显著差异。

the median experiment results. Compared with the GESR-baseline, GESR-opt, GESR-slt, GESR-gradient,and GESR-mutation yields p values below 1e-6, 1e-2, 1e-7, and 1e-6, respectively, indicating significant differences $\left(\mathrm{p}<0.01\right)$ with and without the inclusion of the proposed weight optimization module, semantic point tournament selection module, geometric semantic method, and semantic gradient module respectively.
中位数实验结果。与 GESR 基线相比，GESR-opt、GESR-slt、GESR-gradient 和 GESR-mutation 的 p 值分别低于 1e-6、1e-2、1e-7 和 1e-6，表明在包含或不包含所提出的权重优化模块、语义点锦标赛选择模块、几何语义方法和语义梯度模块时， $\left(\mathrm{p}<0.01\right)$ 存在显著差异。

Due to the nonlinear mapping of semantic space, the importance of each dimension in the sub-semantic space is not equivalent. The absence of semantic gradients impairs the accuracy and effectiveness of the semantic variation module. Therefore, compared to the GESR-baseline, the GESR-gradient without the semantic gradient module exhibits a significant drop in the average $R^2$ metric. Another significant drop in accuracy performance is observed for GESR-mutation, which demonstrates the superiority of the geometric semantic approximation module. In terms of average ranking, GSR-Baseline also shows the better stability over performance across different datasets.The details and roles of each module have been further discussed in Appendix B.
由于语义空间的非线性映射特性，子语义空间中各维度的重要性并不等同。语义梯度的缺失会损害语义变异模块的精度与效能。因此，相较于 GESR 基线模型，未搭载语义梯度模块的 GESR-gradient 在平均 $R^2$ 指标上出现显著下滑。而 GESR-mutation 在准确率表现上的另一重大跌幅，印证了几何语义逼近模块的优越性。就平均排名而言，GSR-Baseline 在不同数据集间也展现出更稳定的性能表现。各模块的具体作用与机理已在附录 B 中深入探讨。

In this work, a novel geometric evolution model is proposed for SR to try to improve the spatial approximation accuracy while ensuring the interpretability. In GESR, a geometric semantic mutation method is introduced as a geometric approximation strategy to search for expressions with optimal accuracy in the semantic space, using a semantic gradient vector as an auxiliary tool for addressing semantic space mapping issues. A semantic-based mutation point selection method is further introduced to efficiently identify sub-expressions that need improvement. Taking into account the inherent advantage of linear combinations within the semantic space, we further incorporate the Levenberg-Marquardt algorithm as a key module for globally adjusting and optimizing expressions constants. The state-of-the-art accuracy performance achieved on both the SRSD and SRBench benchmark datasets demonstrates the effectiveness of our approach in real-world and scientific discovery tasks.
在这项工作中，我们提出了一种新颖的几何演化模型（GESR）以提升符号回归的空间逼近精度，同时确保模型的可解释性。GESR 引入了几何语义变异方法作为几何逼近策略，通过语义梯度向量作为辅助工具解决语义空间映射问题，从而在语义空间中搜索具有最优精度的表达式。进一步提出基于语义的变异点选择方法，高效识别需要改进的子表达式。考虑到语义空间中线性组合的固有优势，我们采用 Levenberg-Marquardt 算法作为核心模块对表达式常量进行全局调整和优化。在 SRSD 和 SRBench 基准数据集上达到的最先进精度性能，验证了该方法在现实场景和科学发现任务中的有效性。

521

525

526

527

528

529

530

531

532

533

534

535

536

537

538

539

Krzysztof Krawiec and Tomasz Pawlak. Approximating geometric crossover by semantic backpropa-gation. In Proceedings of the 15th annual conference on Genetic and evolutionary computation, pp. 941-948, 2013.
Krzysztof Krawiec 与 Tomasz Pawlak。通过语义反向传播近似几何交叉。载于《第 15 届遗传与进化计算年会论文集》，第 941-948 页，2013 年。

William La Cava, Tilak Raj Singh, James Taggart, Srinivas Suri, and Jason H Moore. Learning concise representations for regression by evolving networks of trees. arXiv preprint arXiv:1807.00981, 2018.
William La Cava、Tilak Raj Singh、James Taggart、Srinivas Suri 及 Jason H Moore。通过演化树网络学习回归的简洁表示。arXiv 预印本 arXiv:1807.00981，2018 年。

William La Cava, Thomas Helmuth, Lee Spector, and Jason H. Moore. A Probabilistic and Multi-Objective Analysis of Lexicase Selection and $ϵ$ -Lexicase Selection. Evolutionary Computation,27 (3):377-402, 09 2019. ISSN 1063-6560. doi: 10.1162/evco_a_00224.
William La Cava、Thomas Helmuth、Lee Spector 与 Jason H. Moore。词典选择与 $ϵ$ -词典选择的概率及多目标分析。《进化计算》，27(3):377-402，2019 年 9 月。ISSN 1063-6560。doi: 10.1162/evco_a_00224。

William La Cava, Patryk Orzechowski, Bogdan Burlacu, Fabricio de Franca, Marco Virgolin, Ying Jin, Michael Kommenda, and Jason Moore. Contemporary symbolic regression methods and their relative performance. In J. Vanschoren and S. Yeung (eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1, 2021.
William La Cava、Patryk Orzechowski、Bogdan Burlacu、Fabricio de Franca、Marco Virgolin、Ying Jin、Michael Kommenda 及 Jason Moore。当代符号回归方法及其相对性能。载于 J. Vanschoren 与 S. Yeung 合编的《神经信息处理系统数据集与基准跟踪会议论文集》第 1 卷，2021 年。

Mikel Landajuela, Chak Shing Lee, Jiachen Yang, Ruben Glatt, Claudio P Santiago, Ignacio Aravena, Terrell Mundhenk, Garrett Mulcahy, and Brenden K Petersen. A unified framework for deep symbolic regression. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 33985-33998. Curran Associates, Inc., 2022.
米克尔·兰达胡埃拉、李泽星、杨嘉辰、鲁本·格拉特、克劳迪奥·P·圣地亚哥、伊格纳西奥·阿拉韦纳、特雷尔·蒙登克、加勒特·马尔卡希与布伦登·K·彼得森。《深度符号回归的统一框架》。载于 S. Koyejo、S. Mohamed、A. Agarwal、D. Belgrave、K. Cho 与 A. Oh（编），《神经信息处理系统进展》，第 35 卷，第 33985-33998 页。Curran Associates 公司，2022 年。

Nour Makke and Sanjay Chawla. Interpretable scientific discovery with symbolic regression: a review. Artificial Intelligence Review, 57(1):2, 2024.
努尔·马基与桑杰·查瓦拉。《符号回归的可解释科学发现：一项综述》。《人工智能评论》，57(1):2，2024 年。

Yoshitomo Matsubara, Naoya Chiba, Ryo Igarashi, and Yoshitaka Ushiku. Rethinking symbolic regression datasets and benchmarks for scientific discovery. arXiv preprint arXiv:2206.10540, 2022.
松原义智、千叶直也、五十岚亮与牛久祥孝。《重新思考科学发现中的符号回归数据集与基准》。《arXiv 预印本》，arXiv:2206.10540，2022 年。

Trent McConaghy. FFX: Fast, Scalable, Deterministic Symbolic Regression Technology, pp. 235- 260. Springer New York, New York, NY, 2011. ISBN 978-1-4614-1770-5. doi: 10.1007/ 978-1-4614-1770-5_13.
特伦特·麦康纳吉。《FFX：快速、可扩展、确定性的符号回归技术》，第 235-260 页。Springer 纽约，纽约州，2011 年。ISBN 978-1-4614-1770-5。doi:10.1007/978-1-4614-1770-5_13。

Yi Mei, Qi Chen, Andrew Lensen, Bing Xue, and Mengjie Zhang. Explainable artificial intelligence by genetic programming: A survey. IEEE Transactions on Evolutionary Computation, 27(3): 621-641, 2022.
易梅、齐晨、Andrew Lensen、薛冰与张梦杰。基于遗传编程的可解释人工智能综述。《IEEE 进化计算汇刊》，27(3): 621-641，2022 年。

Jorge J Moré. The levenberg-marquardt algorithm: implementation and theory. In Numerical analysis: proceedings of the biennial Conference held at Dundee, June 28-July 1, 1977, pp. 105-116. Springer, 2006.
Jorge J Moré。Levenberg-Marquardt 算法：实现与理论。收录于《数值分析：1977 年 6 月 28 日-7 月 1 日邓迪会议论文集》，第 105-116 页。Springer 出版社，2006 年。

Terrell Mundhenk, Mikel Landajuela, Ruben Glatt, Claudio P Santiago, Daniel faissol, and Brenden K Petersen. Symbolic regression via deep reinforcement learning enhanced genetic programming seeding. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 24912-24923. Curran Associates, Inc., 2021.
Terrell Mundhenk、Mikel Landajuela、Ruben Glatt、Claudio P Santiago、Daniel Faissol 与 Brenden K Petersen。基于深度强化学习增强遗传编程播种的符号回归。载于 M. Ranzato、A. Beygelzimer、Y. Dauphin、P.S. Liang 与 J. Wortman Vaughan（编），《神经信息处理系统进展》，第 34 卷，第 24912-24923 页。Curran Associates 公司，2021 年。

Quang Uy Nguyen and Thi Huong Chu. Semantic approximation for reducing code bloat in genetic programming. Swarm and Evolutionary Computation, 58:100729, 2020.
Quang Uy Nguyen 与 Thi Huong Chu。语义逼近减少遗传编程中的代码膨胀。《群体与进化计算》，58 卷，100729 号，2020 年。

Tomasz P Pawlak, Bartosz Wieloch, and Krzysztof Krawiec. Semantic backpropagation for designing search operators in genetic programming. IEEE Transactions on Evolutionary Computation, 19(3): 326-340, 2014.
托马斯·P·帕夫拉克、巴托斯·维尔洛赫与克日什托夫·克拉维茨。遗传编程中基于语义反向传播的搜索算子设计。《IEEE 进化计算汇刊》，19(3):326-340，2014 年。

Brenden K Petersen, Mikel Landajuela, T Nathan Mundhenk, Claudio P Santiago, Soo K Kim, and Joanne T Kim. Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients. arXiv preprint arXiv:1912.04871, 2019.
布伦登·K·彼得森、米克尔·兰达朱埃拉、T·内森·蒙登克、克劳迪奥·P·圣地亚哥、苏·K·金与乔安妮·T·金。深度符号回归：通过风险寻求策略梯度从数据中恢复数学表达式。arXiv 预印本 arXiv:1912.04871，2019 年。

Gloria Pietropolli, Luca Manzoni, Alessia Paoletti, and Mauro Castelli. Combining geometric semantic gp with gradient-descent optimization. In European Conference on Genetic Programming (Part of EvoStar), pp. 19-33. Springer, 2022.
格洛丽亚·皮耶特罗波利、卢卡·曼佐尼、阿莱西亚·保莱蒂与毛罗·卡斯泰利。将几何语义遗传编程与梯度下降优化相结合。《欧洲遗传编程会议》（EvoStar 分册），第 19-33 页。斯普林格出版社，2022 年。

In this section, for the convenience of reference and comparison, we provide the short descriptions of the baseline methods, including the 14 original SRBench baseline methods and 5 SRSD baseline methods that used in Section 4. The original papers are refered for the additional details.
在本节中，为便于参考和比较，我们提供了基准方法的简要描述，包括第 4 节中使用的 14 种原始 SRBench 基准方法和 5 种 SRSD 基准方法。更多细节请参阅原始论文。

In Algorithm 1, we provide the detailed pseudocode for describing the framework of GESR. The population and semantic library are initialized in Lines 1-2, and the detailed parameter settings are provided in Appendix C.1. In GESR, Two mutation strategies are employed: geometric semantic approximation as the primary variation method(Line 9), and a random replacement strategy for population exploration(Line 10), both performed probabilistically in each generation. The Levenberg-Marquardt algorithm is periodically executed to adjust the global weight parameters of the expression(Line 5). Subsequently, mutated expressions are executed, and selected of nodes for the next mutation(Line 14). Additionally, at regular intervals, several subtrees from top-t expression trees in the population are randomly selected to supplement the semantic library for transfer learning the useful features (Lines 16-19).
在算法 1 中，我们提供了详细描述 GESR 框架的伪代码。种群与语义库的初始化见第 1-2 行，具体参数设置详见附录 C.1。GESR 采用两种变异策略：作为主要变异方法的几何语义近似（第 9 行），以及用于种群探索的随机替换策略（第 10 行），二者在每一代中均按概率执行。周期性执行 Levenberg-Marquardt 算法以调整表达式的全局权重参数（第 5 行）。随后执行变异表达式，并选择节点进行下一轮变异（第 14 行）。此外，定期从种群中排名前 t 的表达式树中随机选取若干子树，补充至语义库以迁移学习有用特征（第 16-19 行）。

Algorithm 1: Geometric evolution symbolic regression.
算法 1：几何演化符号回归

Input : Symbolic regression problem $\mathcal{P}$ ,consists of tabular $\mathrm{data}\left(X,y\right)$
输入：符号回归问题 $\mathcal{P}$ ，由表格数据 $\mathrm{data}\left(X,y\right)$ 组成

Output : Best fitting expression
输出：最佳拟合表达式

Initialize population $\mathcal{P}$ ;
初始化种群 $\mathcal{P}$ ;

Initialize semantic library ${\mathcal{D}}_s$ ;
初始化语义库 ${\mathcal{D}}_s$ ;

for each iteration $i$ do
对每次迭代 $i$ 执行

if $i$ is equal to the interval of constant optimization then
如果 $i$ 等于常数优化间隔则

$\mathcal{C}\leftarrow$ optimize population $\mathcal{P}$ with Levenberg-Marquardt algorithm
使用 Levenberg-Marquardt 算法优化种群

else  否则

$s\sim U\left(0,1\right)$ ;

if $s<0.95$ then  如果 $s<0.95$ 则

$\mathcal{C}\leftarrow$ approximate the target semantics of $\mathcal{P}$ with geometric semantic mutation
通过几何语义变异近似目标语义

method.  方法。

else  否则

$\mathcal{C}\leftarrow$ population variation with random mutation method.
采用随机变异方法的 $\mathcal{C}\leftarrow$ 种群变异。

end  结束

end  结束

Execute the symbolic tree in $\mathcal{C}$ ;
在 $\mathcal{C}$ 处执行符号树；

Select $\mathcal{P}$ with tournament strategy to form the new population ${\mathcal{P}}^{\prime }$ ;
使用锦标赛策略选择 $\mathcal{P}$ 以形成新种群 ${\mathcal{P}}^{\prime }$ ；

if $i$ is equal to the interval of semantic library update then
如果 $i$ 等于语义库更新的间隔时间则

$\mathcal{T}\leftarrow$ select tok-n symbolic tree from $\mathcal{P}$ ;
从 $\mathcal{P}$ 中选择 tok-n 符号树的 $\mathcal{T}\leftarrow$ ；

update ${\mathcal{D}}_s$ with sub-trees from $\mathcal{T}$ ;
用 $\mathcal{T}$ 的子树更新 ${\mathcal{D}}_s$ ；

end  结束

end  结束

In the geometric semantic mutation strategy, for each expression tree in the population, we first select a subtree to be mutated from several candidate nodes based on Eq 11. Subsequently, a specified number of candidates are randomly selected from the semantic library, and linear combination and filter through Eq 4-10 are performed to replace the mutated subtree. It is worth noting that the representation of the subtree obtained by our geometric semantic method typically takes on the form $k_1{p}_1+k_2{p}_2$ ,When $k_1$ or $k_2$ is 0.0,it yields $k\cdot p$ or 0 . However,this resulting subtree lacks a combined representation of constants and features,i.e., $k_1{p}_1+c$ . Therefore,for each expression tree in the population, we insert an additional constant node " 1 " into the candidate so that this combinatorial subtree can generate such a representation.
在几何语义变异策略中，对于种群中的每个表达式树，我们首先基于公式 11 从若干候选节点中选择一个待变异的子树。随后，从语义库中随机选取指定数量的候选，通过公式 4-10 进行线性组合与筛选，以替换被变异的子树。值得注意的是，通过我们的几何语义方法获得的子树表示通常呈现为 $k_1{p}_1+k_2{p}_2$ 的形式，当 $k_1$ 或 $k_2$ 为 0.0 时，会生成 $k\cdot p$ 或 0。然而，由此产生的子树缺乏常数与特征的组合表示，即 $k_1{p}_1+c$ 。因此，对于种群中的每个表达式树，我们在候选节点中额外插入一个常数节点“1”，以便该组合子树能够生成此类表示。

In the random replacement strategy, two traditional GP methods are employed: random subtree replacement and random node replacement. With a tournament selection strategy in small size, this
在随机替换策略中，采用了两种传统的遗传编程方法：随机子树替换与随机节点替换。配合小规模锦标赛选择策略，该方法能为进化过程提供多样性。

strategy can provide diversity in the evolution. Furthermore, to reduce the tree size, the limited subtree depth generated by the random subtree replacement strategy is less than or equal to the original subtree.
此外，为控制树结构规模，通过随机子树替换策略生成的新子树深度将限制为小于或等于原子树深度。

Table 3: Hyperparameter setting
表 3：超参数设置

The hyperparameters for GESR are listed in Table 4. In this work, our method aims to directly approximate the output of the target expression numerically and then map the operation to the formula in the population. To mitigate the impact of poor initial expression structure, we employ a parameter setting with a large population and small depth, allowing for broader coverage of initial expression structures and increased fault tolerance. A smaller tournament size also facilitates group exploration. For variation rate, we primarily utilize geometric semantic search methods while maintaining a certain probability of random variation. The significance of random variation lies in its facilitation of early-stage population exploration, prevention of local optimization due to the solidification of population expressions in later stages, and selection of new expression structures into the semantic library. The updated pattern for the semantic database involves high-frequency sampling with small amounts to sample elite individuals' expression structures at each stage of evolution. Each semantic database samples 20 subtrees from the top 10 individuals' semantics and randomly replaces these subtrees within the database. During evolution,our function set consists of 8 tuples <+,-,*,%protected,sin, $\cos ,\log ,\exp >$ protected by interval arithmetic Keijzer (2003).
GESR 的超参数列于表 4 中。本工作中，我们的方法旨在直接数值逼近目标表达式的输出，随后将运算映射至种群中的公式。为减轻初始表达式结构不良的影响，我们采用大种群、小深度的参数设置，以广泛覆盖初始表达式结构并提升容错性。较小的锦标赛规模也有利于群体探索。变异率方面，我们主要运用几何语义搜索方法，同时保持一定概率的随机变异。随机变异的意义在于促进早期种群探索，防止后期因种群表达式固化导致的局部优化，并筛选新表达式结构进入语义库。语义数据库的更新模式采用高频少量采样，在进化各阶段抽取精英个体的表达式结构。 每个语义数据库从排名前 10 的个体语义中采样 20 棵子树，并在数据库内随机替换这些子树。进化过程中，我们的函数集由 8 个元组组成<+,-,*,%受保护的,sin, $\cos ,\log ,\exp >$ ，采用区间算术保护（Keijzer，2003）。

In addition, the diversity of the expression tree structure is important. we think employing a large population with fewer generations outperforms the alternative of a small population with numerous generations. That is because of the gradual loss of population diversity as the number of iterations increases, in which the localized mutation is not enough to alter the overall structure. Therefore, we use a large population size with relatively small generations to promote exploration within the population.
此外，表达式树结构的多样性至关重要。我们认为采用大种群配合较少代数优于小种群多代数的方案，因为随着迭代次数增加，种群多样性会逐渐丧失，此时局部突变不足以改变整体结构。因此，我们使用较大种群规模配合相对较少的代数以促进种群内的探索。

The experiments are carried out on a physical machine equipped with an Intel Core(TM) i7-8700 CPU @ 3.20GHz and a single NVIDIA RTX 3090 GPU with 24268-MB global memory.
实验在一台物理机上执行，配置为 Intel Core(TM) i7-8700 CPU @ 3.20GHz 处理器，以及单块 NVIDIA RTX 3090 GPU，其全局显存为 24268MB。

Table 4: Notations  表 4：符号表

In addition to SRBench and SRSD benchmark datasets, our method have also been evaluated on several widely used datasets.
除 SRBench 和 SRSD 基准数据集外，我们的方法还在多个广泛使用的数据集上进行了评估。

The range of the Livermore dataset for both training and test sets is uniformly limited to $\left[-10,10\right]$ , with a total of 1000 sets. The training set is randomly generated within this range, while the test set adopts equal interval sampling. It should be noted that only datasets within the function set's range are selected for evaluation,given that our function set includes <+,-,*,%,sin,cos, $\log$ ,exp>. Each dataset is evaluated 10 times with different seeds.
利弗莫尔数据集的训练集和测试集范围统一限定在 $\left[-10,10\right]$ 内，共 1000 组数据。训练集在此范围内随机生成，测试集则采用等间隔采样。需注意的是，仅选取函数集定义域内的数据集进行评估，因我们的函数集包含<+,-,*,%,sin,cos, $\log$ ,exp>。每组数据使用不同随机种子重复评估 10 次。

Table 5 shows the median $R^2$ performance of our method on each dataset over 10 runs,as well as the success rate under two precision settings. The $Ac{c}_{\tau }$ refers to the accuracy solution rate and $\tau$ indicates the precision. It refers to successfully finding a solution when $1-R^2<1.0e^{-\left(\tau +1\right)}$ . The experimental results exhibit a high accuracy across most of the datasets. However, the GESR on datasets containing protection operators (Livermore-4, Livermore-12), particularly higher-order ones (Livermore-5, Livermore-12), exhibit decreased performance. The decline in fitting ability may be attributed to the interval operation protection strategy. Nevertheless, compared to traditional protection operators, this strategy can better prevent overfitting in expressions featuring numerous
表 5 展示了我们的方法在 10 次运行中各数据集的 $R^2$ 性能中位数，以及两种精度设置下的成功率。其中， $Ac{c}_{\tau }$ 代表准确解率， $\tau$ 表示精度。当 $1-R^2<1.0e^{-\left(\tau +1\right)}$ 时，它指的是成功找到一个解。实验结果显示在大多数数据集上具有较高的准确性。然而，在包含保护算子（Livermore-4、Livermore-12）的数据集上，特别是高阶保护算子（Livermore-5、Livermore-12）的数据集上，GESR 表现出性能下降。拟合能力的下降可能归因于区间操作保护策略。尽管如此，与传统保护算子相比，该策略能更好地防止具有大量特征的表达式出现过拟合。