Jones K.S., Galliers J.R. Evaluating Natural Language Processing Systems: An Analysis and Review

Springer, 1995. — 242 p.This book presents a detailed analysis and review of NLP evaluation, in principle and in practice. Chapter 1 examines evaluation concepts and establishes a framework for NLP system evaluation. This makes use of experience in the related area of information retrieval, and the analysis also refers to evaluation in speech processing. Chapter 2 surveys significant evaluation work done so far, in relation both to particular tasks, for instance machine translation, and to NLP evaluation methodology as such, for example as this bears on generic system evaluation. The conclusion is that evaluation strategies and techniques for NLP need much more development, in particular to take proper account of the influence of system tasks and settings. Chapter 3 develops a general approach to NLP evaluation, aimed at methodologically sound strategies for test and evaluation motivated by comprehensive performance factor identification. The analysis throughout the book is supported by extensive illustrative examples.The Framework: Scope and Concepts
NLP Evaluation: Work and Strategies
Strategies for Evaluation