Parameter Identification for Partial Differential Equations with Spatiotemporal Varying Coefficients
具有时空变化系数的偏微分方程的参数辨识

Parameter identification for partial differential equations (PDEs) is also known as the inverse problem, encompassing various mathematical branches such as numerical analysis, nonlinear analysis, and optimization algorithms. The target of the inverse problem is inferring unknown parameters of PDE from a set of spatiotemporal data with potential noise [1] and this field has progressed rapidly over the past few decades with proposed methods such as the sparse Bayesian learning algorithm [2], the least squares method [3], the frequency and Bayesian methods [4], and the physics-informed neural networks (PINNs) [5], etc. In the mechanics of material fields, accurate property parameter detection will benefit the damage detection and design for new multi-functional materials [6]. In biomechanics, identifying important parameters in human tissue can be helpful for treatment and disease prevention [7, 8]. And parameter identification method is also widely used in other engineering fields such as oil exploration and fluid mechanism [9, 10].
偏微分方程 （PDE） 的参数识别也称为逆问题，包括各种数学分支，例如数值分析、非线性分析和优化算法。逆问题的目标是从一组具有潜在噪声的时空数据中推断出 PDE 的未知参数 [1]，在过去的几十年里，该领域随着提出的方法迅速发展，例如稀疏贝叶斯学习算法 [2]、最小二乘法 [3]、频率和贝叶斯方法 [4] 以及物理信息神经网络 （PINN） [5]等。在材料领域的力学中，精确的性能参数检测将有利于新型多功能材料的损伤检测和设计 [6]。在生物力学中，识别人体组织中的重要参数有助于治疗和疾病预防 [7， 8]。参数辨识方法也广泛应用于石油勘探、流体机构等其他工程领域 [9， 10]。

Nowadays, multi-state systems with time-varying or space-varying parameters have been widely used in fields such as physics, biology, chemical processes [11], and society [12]. One of the most powerful ways of understanding a multi-state complex system with time-varying parameters is discovering its state transition path. Various theories have been proposed to model and characterize the system dynamics such as the transition path theory [13], the transition path sampling [14], and the Markov state model [15]. For the system governed by a varying parametric PDE, the evolutionary process of the varying parameters determines the state transition path of the multi-state system. For space-varying parameters in higher dimensions, the transition region could be inscribed instead of the transition path. As such, identifying the unknown varying parameters is becoming a necessary first step for discovering the pattern variation in complex systems.
如今，具有时变或空间变化参数的多态系统已广泛应用于物理学、生物学、化学过程 [11] 和社会学 [12] 等领域。理解具有时变参数的多态复杂系统的最有效方法之一是发现其状态转换路径。已经提出了各种理论来建模和表征系统动力学，例如过渡路径理论 [13]、过渡路径采样 [14] 和马尔可夫状态模型 [15]。对于由变化参数偏微分方程控制的系统，变化参数的进化过程决定了多态系统的状态转换路径。对于更高维度中的空间变化参数，可以内接过渡区域而不是过渡路径。因此，识别未知的变化参数正在成为发现复杂系统中模式变化的必要第一步。

The PINNs have been demonstrated as an efficient way to infer the unknown parameters of PDEs from the observed data. The original idea of PINNs was introduced by Lagaris in 1998 [16], and has been well established by Raissi et al. for solving two main problems: the forward problem for PDE resolution and parameter identification for PDE [17, 18]. From Raissi, varied numerical techniques have been proposed to improve the performance of PINNs for that two problems [19, 20] and been successfully used in solving problems in materials [21], biology [22], topological optimization [23], and fluid[24]. For the varying parameter inferring task, Revanth et al. proposed the backward compatible PINNs(bc-PINNs)[25] to learn time-varying parameters of time-varying parametric Burgers’ equation from the observed data without any prior information. However, the inferring results of bc-PINNs only follow a trend similar to the true values. As a result, such inaccurate results are insufficient for us to explore the transition path. To solve the above, we need a more accurate parameter identification method.
PINN 已被证明是从观察到的数据中推断 PDE 未知参数的有效方法。PINN 的最初想法由 Lagaris 于 1998 年提出 [16]，并已被 Raissi 等人很好地确立，用于解决两个主要问题：偏微分方程分辨率的正向问题和偏微分方程的参数识别 [17， 18]。Raissi 提出了各种数值技术来提高 PINN 在这两个问题上的性能 [19， 20]，并成功用于解决材料 [21]、生物学 [22]、拓扑优化 [23] 和流体 [24] 中的问题。对于变化参数推断任务，Revanth 等人提出了向后兼容的 PINN（bc-PINN）[25]，以从没有任何先验信息的观测数据中学习时变参数 Burgers 方程的时变参数。但是，bc-PINN 的推理结果仅遵循与真实值相似的趋势。因此，这种不准确的结果不足以让我们探索过渡路径。要解决上述问题，我们需要一种更准确的参数识别方法。

After obtaining the inferring results of the varying parametric PDEs, the next part is detecting the change region of the varying parameters. Change-point detection is an important part of time series analysis and probability anomaly detection [26]. This work requires us to pinpoint the locations of changes in statistical characteristics and points in time at which the probability density functions change [27]. Based on the parameter inferring results of a varying system, a fast and accurate change point detection method may contribute to detecting the change points of the system and locating their position, which may be signification for us to discover the state transition path. For time series data, There has been extensive work in detection change points [28, 29, 30] and becomes a signification part of controlling the reliability and stability of the system. Unlike time series analysis, in this study, the change points of time-varying and space-varying parameters make up the region of variation about the intrinsic nature of the multi-state complex system. It would be interesting research to reveal this hidden parameter variation from the output of the system.

Data-driven statistical modeling based on finite mixture distributions is a rapidly evolving field, with a wide range of applications expanding rapidly [31]. Recently, the finite mixture models is utilized in various fields, such as biometrics, physics, medicine, and marketing. It offers a straightforward method for describing a continuous system’s variation through discrete state space. Despite being a simple linear extension of the classical statistical model, finite mixture models share features concerning inference, specifically a discrete latent structure that results in certain fundamental challenges in estimation, such as the need to determine the unknown number of groups, states, and clusters. The expectation maximization(EM) algorithm is an iterative technique based on maximum likelihood estimation for estimating the parameters of statistical models when the data comprises both observed and hidden variables in the context of finite mixture models [32]. The key advantage of the EM algorithm is that it provides a means of estimating the parameters of models with latent variables without explicitly computing the posterior distribution of the latent variables. This statistical method is particularly useful when there are missing or incomplete data, or when the data is partially observed. This can be computationally efficient, especially when dealing with complex models.

In this paper, we introduce a novel framework for discovering the state transition path of a multi-state parametric PDE system in two steps. Firstly, we use the modified constrained self-adaptive physics-informed neural networks (cSPINNs) to identify the unknown varying parameters and then detect the change region via a change point detection method based on a finite mixture model. Specifically, we modify the cSPINNs by adding a sub-network to learn the varying parameters and this can obtain more accurate results than the previous bc-PINNs. Next, we detect the change points concerning where the parameter change based on the inferring results by employing the finite mixture method. Finally, we take the 1D time-varying parametric Burgers’ Equation and 2D space-varying wave equation as test examples to demonstrate the performance of our method.

This paper is structured as follows. In section 2, we describe forms of parametric partial differential equations with time and space-varying parameters which are the test cases for our method. In section 3, the proposed framework containing two main methods for discovering the state transition path is presented in detail. In section 4, we test the performance of our framework based on the 1D time-varying parametric Burgers’ equation and the 2D space-varying parametric wave equation and analysis their results. Section 5 is the comparison of cSPINNs and bc-PINNs via 1D Burgers’ equation. Section 6 is the performance of our framework on 2D space-varying wave equation and section 7 is the conclusion and discussion.

To elucidate the situation of the partial differential equations with Time-varying parameters, we use the following 1D time-varying parametric Burgers’ equation as an example. The Burgers’ equation is a nonlinear second-order partial differential equation that is used as a simplified model in fluid mechanics. The equation is given by the Dutch mathematician Johannes Burgers’ [33] and in this study, we generally write as the following form

where $u$ is the fluid velocity at position $x$ and time $t$, the term $\lambda_{1}uu_{x}$ is known as the convective term, the term $\lambda_{2}u_{xx}$ is the diffusive term and $\lambda_{2}$ is the kinematic viscosity of the fluid. The Burgers’ equation combines the effects of convection and diffusion in a non-linear way and is used to model a variety of phenomena in fluid mechanics, including shock waves, turbulence, and flow in porous media [34]. Besides fluid mechanics, it has also been used in other areas of physics, such as in modeling traffic flow in transportation engineering [35].

Let $\lambda_{1}$ and $\lambda_{2}$ be time-varying parameters and take values in a finite discrete parameter space. We rewrite equation (2.1) as

Thus we get a continuous system with discrete states. In this time-varying parameter system, the parameter may exhibit local invariance. As such, in the global time domain, research attention is directed toward how the system state changes over time and at which points these changes occur. The subsequent objective of this study is to establish a comprehensive mathematical framework that builds upon existing solutions of the system. This framework serves to address the inverse problem for parameters in the equation and change point detection of time-varying parameter systems.

In this paper, the observed data of the 1D time-varying parametric Burgers’ equation is computed via the numerical method fast Fourier transform where the initial value is as givens:

and the domain is $(x,t)\in[-8,8]\times(0,10]$. The observed data of three cases: constant parameters without change point, only $\lambda_{1}$ changes once, and $\lambda_{1}$ and $\lambda_{2}$ are all change with multiple change points are shown in the figure 1.

From the above figures, it is clear that the transition path of states and the change points of the time-varying system can not be revealed directly from the observed data. Moreover, the difference between the first and the second is obscure, let alone the system information and state transfer paths. Therefore, we can apply the modified cSPINNs as a bridge to link the observed data and the unknown parameters. In this way, we can discover the hidden information together with the transition path.

For the situation of partial differential equations with space-varying parameters, as a contrast to that previous example, we will introduce the 2D wave equation, whose parameters are not a constant in the space plane. The wave equation is a mathematical model that describes wave phenomena. It is typically expressed as a partial differential equation and can describe wave processes in both space and time. It has widespread applications in physics, engineering, mathematics, and other fields. The general form of the wave equation can be written as:

Here, the $u$ represents the wave amplitude, the $\alpha$ is the wave speed, and the $\nabla^{2}$ is the Laplacian operator, which represents the second derivative in space. This equation describes how the wave amplitude changes and propagates during a wave process. The second time derivative represents the acceleration of the wave amplitude, while the Laplacian operator represents the second derivative in space. Let $\alpha$ be a space-varying parameter $\alpha(x,y)$ and then rewrite the (2.4) as

In many cases of scientific computing research, there can be sudden and discontinuous changes in local regions, which can have a significant impact on the output of the system. Therefore, it is of great practical significance to obtain the regions of varying neutral states in such a space through scientific calculations. By doing so, we can better understand the underlying physical processes and develop more accurate models to describe them.

In this section, we use two parts to introduce our state transition path discovery framework for a varying parameter system. Firstly, we illustrate the modified cSPINNs method for identifying varying parameters from the observed data. Next, we describe the finite mixture model as our change point detection method.

In this section, we use three distinctive types of numerical cases to test the performance of our framework. Moreover, those three category cases represent different evolutionary models of the time-varying 1D parametric Burgers’ equation, and their hidden state transition paths can be discovered via our framework.

To better identify parameter $\lambda_{1}(t)$, a sub-network with the input $t$ and the output $\lambda_{1}(t;\phi)$ is used to model the dynamics of the parameter, where $\phi$ denotes all trainable parameters of the network and could be optimized during training with the time-varying parameters $\lambda_{1}$ and $\lambda_{2}$. The loss function is denoted as

(a). Mean squared error on the observed data

(b). Mean squared error of the residual points

(c). Total mean squared error for inverse

In the following numerical experiments, we will get $N_{o}=4,000$ observed data, and $N_{r}=64,000$ residual points randomly sampled from the computational domain with $\Omega=[-8,8],T=10$. We let the weights of the PDE loss term and the residual loss term $\lambda_{o}=\lambda_{R}=1$. We use the modified multilayer perceptron (MLP) [39] with a depth of 6, a width of 128, and the tahn activation function as the Neural Network 1 for solving the inverse problem. As for Neural Network 2, a modified MLP is used here, which has 1 input neuron and consists of 4 hidden layers with 40 neurons in each layer, and the activation function is chosen as tanh. The Adam optimizer is used here to minimize the loss function with $N_{e}=200,000$ epochs. Meanwhile, We set the batch size of residual points $N_{bs}=4,000$ to reduce the memory requirement of hardware. The initial learning rate is 0.001, and the exponential learning rate annealing method is applied here with hyper-parameter $\gamma=0.9$ during training. The total time-domain $[0,10]$ of the parametric Burgers’ equation has been discretized into 256 times steps uniformly. To identify the parameters $\lambda_{1}$ and $\lambda_{2}$, all the observed data within five steps segment has been chosen. Prediction errors of identifying parameters via modified cSPINNs are shown in the appendix Appendix B: Absolute Error between Reference and Predicted Solution of 1D parametric Burgers’ Equation while the statistical inferring results and the $L^{2}$ error for learning Burgers’ equation are shown in the table 1. Next, we start to exhibit our results.