  Recently, researchers have utilized neural networks to accurately solve
partial differential equations (PDEs), enabling the mesh-free method for
scientific computation. Unfortunately, the network performance drops when
encountering a high nonlinearity domain. To improve the generalizability, we
introduce the novel approach of employing multi-task learning techniques, the
uncertainty-weighting loss and the gradients surgery, in the context of
learning PDE solutions. The multi-task scheme exploits the benefits of learning
shared representations, controlled by cross-stitch modules, between multiple
related PDEs, which are obtainable by varying the PDE parameterization
coefficients, to generalize better on the original PDE. Encouraging the network
pay closer attention to the high nonlinearity domain regions that are more
challenging to learn, we also propose adversarial training for generating
supplementary high-loss samples, similarly distributed to the original training
distribution. In the experiments, our proposed methods are found to be
effective and reduce the error on the unseen data points as compared to the
previous approaches in various PDE examples, including high-dimensional
stochastic PDEs.
