Bayesian Probabilistic Numerical Methods for Ordinary and Partial Differential Equations

Abstract

Differential equations provide an important mathematical framework for modelling the behaviour of quantities that evolve in continuous time and space, often within a complex system or process. Many physical phenomena, including fundamental laws such as Newton’s laws of motion, are formulated as differential equations. However, most useful differential equations lack a closed form solution expressible in terms of established functions, and so in practice numerical methods are required to obtain a discrete approximation to quantities of interest. Classical numerical methods approximate quantities of interest by taking a finite number of evaluations from some known and computationally tractable quantity, such as the gradient field, and use these within an algorithm to construct an approximation. This is similar to statistics, where a finite number of observations of some unknown, underlying process are used to infer the process itself. In this view, numerical algorithms can be interpreted as estimators, and statistical considerations can be brought to bear. Going further, on can consider probabilistic numerical methods, which output a probability distribution over the quantity of interest. In recent years, this idea has emerged into a new field of research, called Probabilistic Numerics. In the first part of this thesis, an exact Bayesian probabilistic numerical method for ordinary differential equations (ODEs) is presented. The method is a synthesis of classical Lie group theory, to exploit underlying symmetries in the gradient field, and non-parametric regression in a transformed solution space for the ODE. The procedure is presented in detail for first and second order ODEs and relies on a certain strong technical condition – existence of a solvable Lie algebra – being satisfied. Numerical illustrations are provided for nonlinear first and second order ODEs. However, the ability to perform exact Bayesian inference comes at a high price, because the class of ODEs that admit a solvable Lie algebra is limited. In the second part of this thesis, an approximate Bayesian probabilistic numerical method for nonlinear partial differential equations (PDEs) is presented. A Bayesian treatment of nonlinear PDEs does not yet exist, as the case of nonlinear PDEs poses substantial challenges from an inferential perspective, most notably due to the absence of explicit conditioning formula. This thesis extends earlier work on linear PDEs to a general class of initial value problems specified by nonlinear PDEs. Numerical experiments are conducted on a range of examples, and indicate the proposed method is able to provide meaningful probabilistic uncertainty quantification for the unknown solution of the PDE, while controlling the number of times the right-hand-side of the PDE is evaluated. This is practically useful in situations where evaluation of the right-hand-side of the PDE is associated with a high computational cost. The nascent field of Probabilistic Numerics is receiving increased attention, but fundamental questions remain regarding aims and scope of the field. The contributions of this thesis, while limited to proofs of concept, are helpful in clarifying a role for Bayesian statistics in the probabilistic solution of differential equations. The thesis concludes with a discussion, which is broadly supportive of taking a Bayesian approach to differential equations, whilst highlighting where exact Bayesian inference may not be achievable and suggesting approximation strategies in that context