Linear Dimensionality Reduction

This book provides an overview of some classical linear methods in Multivariate Data Analysis. This is an old domain, well established since the 1960s, and refreshed timely as a key step in statistical learning. It can be presented as part of statistical learning, or as dimensionality reduction with a geometric flavor. Both approaches are tightly linked: it is easier to learn patterns from data in low-dimensional spaces than in high-dimensional ones. It is shown how a diversity of methods and tools boil down to a single core method, PCA with SVD, so that the efforts to optimize codes for analyzing massive data sets like distributed memory and task-based programming, or to improve the efficiency of algorithms like Randomized SVD, can focus on this shared core method, and benefit all methods. This book is aimed at graduate students and researchers working on massive data who have encountered the usefulness of linear dimensionality reduction and are looking for a recipe to implement it. It has been written according to the view that the best guarantee of a proper understanding and use of a method is to study in detail the calculations involved in implementing it. With an emphasis on the numerical processing of massive data, it covers the main methods of dimensionality reduction, from linear algebra foundations to implementing the calculations. The basic requisite elements of linear and multilinear algebra, statistics and random algorithms are presented in the appendix.