Biau G., Devroye L. Lectures on the Nearest Neighbor Method

Cham: Springer International Publishing, 2015. — 284 p.This text presents a wide-ranging and rigorous overview of nearest neighbor methods, one of the most important paradigms in machine learning. Now in one self-contained volume, this book systematically covers key statistical, probabilistic, combinatorial and geometric ideas for understanding, analyzing and developing nearest neighbor methods.
Gérard Biau is a professor at Université Pierre et Marie Curie (Paris). Luc Devroye is a professor at the School of Computer Science at McGill University (Montreal).Order statistics and nearest neighbors
The nearest neighbor distance
The k-nearest neighbor density estimate
Uniform consistency
Weighted k-nearest neighbor density estimates
Local behavior
Entropy estimationThe nearest neighbor regression function estimate
The 1-nearest neighbor regression function estimate
L p -consistency and Stone’s theorem
Pointwise consistency
Uniform consistency
Advanced properties of uniform order statistics
Rates of convergence
Regression: the noiseless case
The choice of a nearest neighbor estimateBasics of classification
The nearest neighbor rule: fixed k
The nearest neighbor rule: variable k
Appendix