Regression loss
Regression loss measures the difference between ANN prediction and observed data, including the initial data of PDE, but can be more general and provides much flexibility.
If the Riemann IVP \(u_0(x)=1-\mathrm{Heaviside}(x)\) is posed as initial, the exact solution to the inviscid Burger's equation with Riemann IVP is \[ u(t,x) = u_0(x-t/2) \]
then the corresponding regression loss can be chosen as \[ L_\mathrm{reg}(\theta)\propto \int_{\Omega} |u(0,x;\theta) - u_0(x)|^2 \mathrm{d}x. \]
A minimization on total loss \[ L(\theta) = L_\mathrm{res}(\theta) + L_\mathrm{reg}(\theta) \] leads to the follow approximate solutions.
| Shallow PINN solution 2D View | Shallow PINN solution 3D View |
|---|---|
 |  |
Increasing \(\dim\theta\) can lead to a more accurate solution.
| Deep PINN solution 2D View | Deep PINN solution 3D View |
|---|---|
 |  |
This setup can be much more flexible in the observed data. Suppose the initial data \(u_0(x)\) in the Riemann IVP are not observed. Instead, a moving sensor is placed in the domain of interest to measure the gas pressure. (Inviscid Burger's equation for ideal gas dynamics) We can control the motion of the sensor to be \[ x(t)=\frac{1+\sin(4\pi t)}{4}, \] and the observed data from the sensor becomes \[ u(t, x(t)) = f(t) \]
In this case, the regression loss is redefined as \[ L_\mathrm{reg}(\theta)\propto \int_{0}^{1} |u(t,x(t);\theta) - f(t)|^2 \mathrm{d}t. \] Minimizing the same total loss \(L(\theta) = L_\mathrm{res}(\theta) + L_\mathrm{reg}(\theta)\), we can recover the full solution on the space-time domain of interest.
The recovered solution can achieve \[ \|u(t,x;\theta)-u_\mathrm{exact}(t,x)\|_{L^2([0,1]\times[-1,1])} = 1.16\times10^{-2}. \]
As a mesh-free method, PINN can overcome the difficults for traditional methods (e.g. finite difference) caused by data misalignment.
\begin{align} u_t + u u_x &= 0 \\ \frac{u(t+\Delta t, x) - u(t, x)}{\Delta t} + u(t,x)\cdot \frac{u(t, x+\Delta x) - u(t, x-\Delta x)}{2\Delta x} &= 0 \end{align}
The reasons to distinguish regression loss from residual loss are
- regression loss can be evaluated without back-propagation, while residual loss cannot
- if there's limited number of observed data, regression loss may place constrains on model complexity (\(\dim\theta\))