Powell W.B., Ryzhov I.O. Optimal Learning

John Wiley & Sons, 2012. — 404 p. — ISBN: 978-0470596692.Learn the science of collecting information to make effective decisions.
Everyday decisions are made without the benefit of accurate information. Optimal Learning develops the needed principles for gathering information to make decisions, especially when collecting information is time-consuming and expensive. Designed for readers with an elementary background in probability and statistics, the book presents effective and practical policies illustrated in a wide range of applications, from energy, homeland security, and transportation to engineering, health, and business.
This book covers the fundamental dimensions of a learning problem and presents a simple method for testing and comparing policies for learning. Special attention is given to the knowledge gradient policy and its use with a wide range of belief models, including lookup table and parametric and for online and offline problems. Three sections develop ideas with increasing levels of sophistication:1) Fundamentals explores fundamental topics, including adaptive learning, ranking and selection,
the knowledge gradient, and bandit problems.
2) Extensions and Applications features coverage of linear belief models, subset selection models,
scalar function optimization, optimal bidding, and stopping problems.
3) Advanced Topics explores complex methods including simulation optimization,
active learning in mathematical programming, and optimal continuous measurements.Each chapter identifies a specific learning problem, presents the related, practical algorithms for implementation, and concludes with numerous exercises. A related website features additional applications and downloadable software, including MatLAB and the Optimal Learning Calculator, a spreadsheet-based package that provides an introduc­tion to learning and a variety of policies for learning.The Challenges of Learning.
Learning the Best Path, Areas of Application, Major Problem Classes, The Different Types of Learning,
Learning from Different Communities, Information Collection Using Decision Trees, Website and Downloadable Software, Goals of this Book.
Adaptive Learning.
The Frequentist View, The Bayesian View, Updating for Non-Gaussian Priors, Monte Carlo Simulation, Why Does It Work?
The Economics of Information.
An Elementary Information Problem, The Marginal Value of Information, An information Acquisition Problem.
Ranking and Selection.
The Model, Measurement Policies, Evaluating Policies, More Advanced Topics.
The Knowledge Gradient.
The Knowledge Gradient for Independent Beliefs, The Value of Information and the S-Curve Effect,
Knowledge Gradient for Correlated Beliefs, Anticipatory Versus Experiential Learning,
The Knowledge Gradient for Some Non-Gaussian Distributions, Relatives of the Knowledge Gradient,
The Problem of Priors, Discussion, Why Does It Work?
Bandit Problems.
The Theory and Practice of Gittins Indices, Variations of Bandit Problems,
Upper Confidence Bounding, The Knowledge Gradient for Bandit Problems.
Elements of a Learning Problem.
The States of our System, Types of Decisions, Exogenous Information, Transition Functions, Objective Functions, Evaluating Policies.
Linear Belief Models.
Applications: Maximizing Ad Clicks, Dynamic Pricing, Housing Loans, Optimizing Dose Response;
A Brief Review of Linear Regression, The Knowledge Gradient for a Linear Model, Application to Drug Discovery, Application to Dynamic Pricing.
Subset Selection Problems.
Applications, Choosing a Subset Using Ranking and Selection, Larger Sets, Very Large Sets.
Optimizing a Scalar Function.
Deterministic Measurements, Stochastic Measurements.
Optimal Bidding.
Modeling Customer Demand, Bayesian Modeling for Dynamic Pricing, Bidding Strategies, Why Does It Work?
Stopping Problems.
Sequential Probability Ratio Test, The Secretary Problem.
Active Learning in Statistics.
Deterministic Policies, Sequential Policies for Classification, A Variance-Minimizing Policy, Mixtures of Gaussians.
Simulation Optimization.
Indifference Zone Selection, Optimal Computing Budget Allocation, Model-Based Simulated Annealing, Other Areas of Simulation Optimization.
Learning in Mathematical Programming.
Applications: Piloting a Hot Air Balloon, Optimizing a Portfolio, Network Problems, Discussion;
Learning on Graphs, Alternative Edge Selection Policies, Learning Costs for Linear Programs.
Optimizing Over Continuous Measurements.
The Belief Model, Sequential Kriging Optimization, The Knowledge Gradient for Continuous Parameters,
Efficient Global Optimization, Experiments, Extension to Higher-Dimensional Problems.
Learning With a Physical State.
Introduction to Dynamic Programming, Some Heuristic Learning Policies, The Local Bandit Approximation,
The Knowledge Gradient in Dynamic Programming, An Expected Improvement Policy.