Scaling limits in computational Bayesian inversion

Schillings, Claudia; Schwab, Christoph

ESAIM: Mathematical Modelling and Numerical Analysis , Tome 50 (2016) no. 6, pp. 1825-1856.

Computational Bayesian inversion of operator equations with distributed uncertain input parameters is based on an infinite-dimensional version of Bayes’ formula established in M. Dashti and A.M. Stuart [Handbook of Uncertainty Quantification, edited by R. Ghanem, D. Higdon and H. Owhadi. Springer (2015).] and its numerical realization in C. Schillings and Ch. Schwab [Inverse Problems 29 (2013) 065011; Inverse Problems 30 (2014) 065007.] Based on the sparsity of the posterior density shown in C. Schillings and Ch. Schwab [Inverse Problems 29 (2013) 065011; Inverse Problems 30 (2014) 065007.]; C. Schwab and A.M. Stuart [Inverse Problems 28 (2012) 045003.], dimension-adaptive Smolyak quadratures can afford higher convergence rates than MCMC in terms of the number $M$ of solutions of the forward (parametric operator) equation in C. Schillings and Ch. Schwab [Inverse Problems 29 (2013) 065011; Inverse Problems 30 (2014) 065007.]. The error bounds and convergence rates obtained in C. Schillings and Ch. Schwab [Inverse Problems 29 (2013) 065011; Inverse Problems 30 (2014) 065007.] are independent of the parameter dimension (in particular free from the curse of dimensionality) but depend on the (co)variance $\Gamma >0$ of the additive, Gaussian observation noise as $exp\left(b\Gamma {}^{-1}\right)$ for some constant $b>0$. It is proved that the Bayesian estimates admit asymptotic expansions as $\Gamma \downarrow 0$. Sufficient (nondegeneracy) conditions for the existence of finite limits as $\Gamma \downarrow 0$ are presented. For Gaussian priors, these limits are shown to be related to MAP estimators obtained from Tikhonov regularized least-squares functionals. Quasi-Newton (QN) methods with symmetric rank-1 updates are shown to identify the concentration points in a non-intrusive way, and to obtain second order information of the posterior density at these points. Based on the theory, two novel computational Bayesian estimation algorithms for Bayesian estimation at small observation noise covariance $\Gamma >0$ with performance independent of $\Gamma \downarrow 0$ are proposed: first, dimension-adaptive Smolyak quadrature from C. Schillings and Ch. Schwab [Inverse Problems 29 (2013) 065011; Inverse Problems 30 (2014) 065007.] combined with a reparametrization of the parametric Bayesian posterior density near the MAP point (assumed unique) and, second, generalized Richardson extrapolation to the limit of vanishing observation noise variance. Numerical experiments are presented which confirm $\Gamma$-independent convergence of the curvature-rescaled, adaptive Smolyak algorithm. Dimension truncation of the posterior density is justified by a general compactness result for the posterior’s Hessian at the MAP point.

S. Agapiou, O. Papaspiliopoulos, D. Sanz-Alonso and A.M. Stuart, Importance Sampling: Computational Complexity and Intrinsic Dimension. Preprint (2015). | arXiv | MR

M. Atyiah, Resolution of Singularities and Division of Distributions. Comm. Pure Appl. Math. 23 (1970) 145–150. | DOI | MR | Zbl

T. Bui-Thanh and O. Ghattas, Analysis of the Hessian for Inverse Scattering Problems. Part II: Inverse Medium Scattering of Acoustic Waves. Inverse Problems 28 (2012) 055002. | DOI | MR | Zbl

T. Bui-Thanh, C. Burstedde, O. Ghattas, J. Martin, G. Stadler and L.C. Wilcox, Extreme-scale UQ for Bayesian Inverse Problems Governed by PDE. ACM/IEEE Supercomputing SC12, Gordon Bell Prize Finalist. Salt Lake City, Utah (2012).

J. Charrier, Strong and Weak Error Estimates for the Solutions of Elliptic PDEs with random coefficients. SIAM J. Numer. Anal. 50 (2013) 216–246. | DOI | MR | Zbl

P. Chen and Ch. Schwab, Sparse-Grid, Reduced-Basis Bayesian Inversion: Nonaffine-Parametric Nonlinear Equations. J. Comput. Phys. 316 (2016) 470–503. | DOI | MR | Zbl

A. Chkifa, A. Cohen and C. Schwab, Breaking the Curse of Dimensionality in Sparse Polynomial Approximation of Parametric PDEs. J. Math. Pures Appliquées 103 (2015) 400–428. | DOI | MR | Zbl

A. Cohen, R. Devore and C. Schwab, Analytic Regularity and Polynomial Approximation of Parametric and Stochastic Elliptic PDEs. Anal. Appl. 9 (2011) 1–37. | DOI | MR | Zbl

A. Conn, N. Gould and P. Toint, Global Convergence of a Class of Trust Region Algorithms for Optimization with Simple Bounds. SIAM J. Numer. Anal. 25 (1988) 433–460. | DOI | MR | Zbl

A. Conn, N. Gould and P. Toint, Convergence of Quasi-Newton Matrices Generated by the Symmetric Rank-One Update. Math. Program. 50 (1991) 177–195. | DOI | MR | Zbl

T. Cui, J. Martin, Y.M. Marzouk, A. Solonen and A. Spantini, Likelihood-Informed Dimension Reduction for Nonlinear Inverse Problems. Inverse Problems 30 (2014) 114015. | DOI | MR | Zbl

T. Cui, K.J.H. Law and Y.M. Marzouk, Dimension-Independent Likelihood-Informed MCMC. J. Comput. Phys. 304 (2016) 109–137. | DOI | MR | Zbl

M. Dashti and A.M. Stuart, The Bayesian Approach to Inverse Problems. Handbook of Uncertainty Quantification, edited by R. Ghanem, D. Higdon and H. Owhadi. Springer (2015). | MR

M. Dashti, K. Law, A.M. Stuart and J. Voss, MAP Estimators and their Consistency in Bayesian Nonparametric Inverse Problems. Inverse Problems 29 (2013) 095017. | DOI | MR | Zbl

J. Dick, T.Q. Legia and Ch. Schwab, Higher Order Quasi Monte Carlo Integration for Holomorphic, Parametric Operator Equations. J. Uncertainty. Quantification 4 (2016) 48–79. | DOI | MR

M.V. Fedoryuk, Asimptotika: integraly i ryady. Spravochnaya Matematicheskaya Biblioteka. [Mathematical Reference Library] (in russian). Nauka Publ., Moscow, Russia (1987). | Zbl

R. Fletcher, An Overview of Unconstrained Optimization in Algorithms for Continuous Optimization, edited by E. Spedicato. Springer, The Netherlands (1994) 109–14. | MR | Zbl

R.P. Ge and M.J.D. Powell, The Convergence of Variable Metric Matrices in Unconstrained Optimization. Math. Program. 27 (1983) 123–143. | DOI | MR | Zbl

M.B. Giles and N.A. Pierce, An Introduction to the Adjoint Approach to Design. Flow, Turbulence and Combustion 65 (2000) 393–415. | DOI | Zbl

M. Girolami and B. Calderhead, Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods. J. R. Stat. Soc.: Series B 73 (2011) 123–214. | DOI | MR | Zbl

M. Hansen and C. Schwab, Analytic Regularity and Best $N$-Term Approximation of High-Dimensional Parametric Initial Value Problems. Vietnam J. Math. 41/2 (2012) 181–215. | MR | Zbl

V.H. Hoang and C. Schwab, Analytic Regularity and Polynomial Approximation of Stochastic, Parametric Elliptic Multiscale PDEs. Anal. Appl. 11 (2013) 1350001. | DOI | MR | Zbl

J. Kaipio and E. Somersalo, Statistical and Computational Inverse Problems. Vol. 160 of Appl. Math. Sci. Springer (2005). | MR | Zbl

B.T. Knapik, A. W. Van Der Vaart and J.H. Van Zanten, Bayesian Inverse Problems with Gaussian Priors. Ann. Stat. 39 (2011) 2626–2657. | DOI | MR | Zbl

K.J.H. Law, Proposals which speed up Function-Space MCMC. J. Comput. Appl. Math. 262 (2014) 127–138. | DOI | MR | Zbl

J. Martin, L.C. Wilcox, C. Burstedde and O. Ghattas and A. Stochastic, Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion. SIAM J. Sci. Comput. 34 (2012) A1460–A1487. | DOI | MR | Zbl

T.A. Moselhy and Y.M. Marzouk, Bayesian Inference with Optimal Maps. J. Comput. Phys. 231 (2012) 7815–7850. | DOI | MR | Zbl

J. Nocedal and S.J. Wright, Numerical Optimization. Springer, 2nd Edition (1999). | MR | Zbl

C. Schillings and Ch. Schwab, Sparse, adaptive Smolyak Quadrature for Bayesian inverse problems. Inverse Problems 29 (2013) 065011. | DOI | MR | Zbl

C. Schillings and Ch. Schwab, Sparsity in Bayesian inversion of parametric operator equations. Inverse Problems 30 (2014) 065007. | DOI | MR | Zbl

C. Schillings and Ch. Schwab, Scaling Limits in Computational Bayesian Inversion. Report 2014-26, Seminar for Applied Mathematics.ETH Zürich, http://www.sam.math.ethz.ch (2014). | Numdam | MR

C. Schillings, M. Sunnåker, J. Stelling and Ch. Schwab, Efficient Characterization of Parametric Uncertainty of Complex (Bio)chemical Networks. PLOS Comp. Biol. (2015), . | DOI

C. Schwab and A.M. Stuart, Sparse deterministic approximation of Bayesian inverse problems. Inverse Problems 28 (2012) 045003. | DOI | MR | Zbl

A. Sidi, On a generalization of the Richardson extrapolation process. Numer. Math. 57 (1990) 365–377. | DOI | MR | Zbl

A.M. Stuart, Inverse problems: a Bayesian approach. Acta Numerica 19 (2010) 451–559. | DOI | MR | Zbl

L. Tierney and J.B. Kadane, Accurate Approximations for Posterior Moments and Marginal Densities. J. Am. Stat. Assoc. 81 (1986) 82–86. | DOI | MR | Zbl