I consider the problem of estimating a n x n covariance matrix from a set of m sampled vectors, each of dimension n. The caveat is that m is much less than n, i.e., the dimension is huge and the number of samples is small. I will explain why this seemingly mundane problem is so important across all of Earth science and then report how this problem has been partially solved in numerical weather prediction (NWP) via a procedure called "covariance localization." We will then construct a theory of optimal localization. A numerical and analytical comparison of optimal and practical approaches reveals that (i) the practical approach, used for decades in NWP, very much mimics an optimal approach, implying that there is mathematical support for the practical, ad hoc solution; and (ii) the details of how covariance matrices are localized have only a second order effect, implying that "even bad localization is good localization." The latter point is quite subtle, but has practical implications which I will discuss. This talk summarizes work done in collaboration with Dr. Daniel Hodyss of the Naval Research laboratory and is supported by an ONR grant (N00014-21-1-2309).

Matthias Morzfeld Website:  https://igppweb.ucsd.edu/~mmorzfeld/