Kuhn M., Johnson K. Feature Engineering and Selection - A Practical Approach for Predictive Models

Boca Raton: CRC Press, 2021. — 314 p.The process of developing predictive models includes many stages. Most resources focus on the modeling algorithms but neglect other critical aspects of the modeling process. This book describes techniques for finding the best representations of predictors for modeling and for nding the best subset of predictors for improving model performance. A variety of example data sets are used to illustrate the techniques along with R programs for reproducing the results.A Simple Example
Important Concepts
A More Complex Example
Feature Selection
An Outline of the Book
Computing
Illustrative Example: Predicting Risk of Ischemic Stroke
Splitting
Preprocessing
Exploration
Predictive Modeling Across Sets
Other Considerations
Computing
A Review of the Predictive Modeling Process
Illustrative Example: OkCupid Profile Data
Measuring Performance
Data Splitting
Resampling
Tuning Parameters and Overfitting
Model Optimization and Tuning
Comparing Models Using the Training Set
Feature Engineering Without OverfittingComputing
Exploratory Visualizations
Introduction to the Chicago Train Ridership Data
Visualizations for Numeric Data: Exploring Train Ridership Data
Visualizations for Categorical Data: Exploring the OkCupid Data
Post Modeling Exploratory VisualizationsComputing
Encoding Categorical Predictors
Creating Dummy Variables for Unordered Categories
Encoding Predictors with Many Categories
Approaches for Novel Categories
Supervised Encoding Methods
Encodings for Ordered Data
Creating Features from Text Data
Factors versus Dummy Variables in Tree-Based ModelsComputing
Engineering Numeric Predictors
Transformations
Many Transformations
Many: Many TransformationsComputing
Detecting Interaction Effects
Guiding Principles in the Search for Interactions
Practical Considerations
The Brute-Force Approach to Identifying Predictive Interactions
Approaches when Complete Enumeration is Practically Impossible
Other Potentially Useful ToolsComputing
Handling Missing Data
Understanding the Nature and Severity of Missing Information
Models that are Resistant to Missing Values
Deletion of Data
Encoding Missingness
Imputation methods
Special CasesComputing
Working with Profile Data
Illustrative Data: Pharmaceutical Manufacturing Monitoring
What are the Experimental Unit and the Unit of Prediction?
Reducing Background
Reducing Other Noise
Exploiting Correlation
Impacts of Data Processing on ModelingComputing
Feature Selection Overview
Goals of Feature Selection
Classes of Feature Selection Methodologies
Effect of Irrelevant Features
Overfitting to Predictors and External Validation
A Case Study
Next Steps
Computing
Greedy Search Methods
Illustrative Data: Predicting Parkinson’s Disease
Simple Filters
Recursive Feature Elimination
Stepwise SelectionComputing
Global Search Methods
Naive Bayes Models
Simulated Annealing
Genetic Algorithms
Test Set ResultsComputing