Among the most exciting developments over the last two decades is the rapid advances in randomized numerical linear algebra (RNLA). This has caused a paradigmatic shift in the computational sciences, and has enabled matrix computations of unprecedented scale. In addition to speed, RNLA often provides solutions with exceptional accuracy and robustness. Success stories in RNLA include low-rank approximation, least-squares problems, and trace estimation. In addition, the field has witnessed recent progress in linear systems, eigenvalue problems, and tensor approximation. Despite these success stories, there is often a wide gap between theory and practice. To illustrate, consider the task of "subspace sketching", which is a key idea in RNLA. There is theory to support a claim that structured embeddings (aka "fast Johnson-Lindenstrauss transforms") based on FFTs provide a geometry preserving subspace embedding if O(dlogd) samples are used for a d-dimensional subspace. However, this is usually a pessimistic estimate, and O(d), say 2d samples, is known to suffice in most problems that arise in practice. A similar story holds for sparse sketches, for which beautiful theory is available, but in practice they often perform much better for a particular instance. The same is true even of deterministic algorithms, such as the column-pivoted QR factorization, specifically whether it gives a rank-revealing QR factorization. While the worst-case analysis suggests the algorithm can be exponentially unstable, decades of practical computation has shown that such examples are vanishingly rare. Clearly, in any of these problems, it would be highly useful to have a methodology to assess the correctness of a solution computed for the particular problem at hand.
The objective of this proposal is to develop techniques for computing upper bounds on the error incurred by a particular instantiation of a randomized algorithm for solving a linear algebraic problem. Such bounds would allow users to combine the computational speed of randomized algorithms with the reliability of classical deterministic methods. It would also help users in safely balancing the needs of speed and precision.
The approach we propose to follow here is close in spirit to the notion of "responsibly reckless" algorithms, recently coined by Jack Dongarra, the latest Turing Award laureate. The idea is to try a fast, but potentially unstable, algorithm, to obtain a potential solution. Then we assess the quality of the solution in a reliable fashion. Specifically, the purpose of this proposal is to develop fast and robust algorithms for this assessment, thereby certifying the correctness of the solution, or otherwise output a warning that the solution may be inaccurate. We will design algorithms, develop theory, and implement them in open-source software optimised for modern computing platforms.
The historical development of finite element methods would help put our project in context. A priori analysis came first, and error bounds that were invaluable in guiding the design of finite element spaces were proven. However, these bounds were too pessimistic to usefully assess the error incurred in any particular instantiation, since the bound had to cover the most adversarial input. A posteriori analysis then emerged in response, as it applies specifically to a problem at hand and can, by drawing on quantities available post-computation, provide accurate estimates or bounds on the error. In this project we aim to achieve the same in the RNLA context.
Specific directions we will pursue include: Subspace embedding, low-rank approximation, column-pivoted QR factorization, linear systems, eigenvalue problems and SVD, least-squares problems, and building Krylov subspaces efficiently.
|