Lynch S.M. Introduction to Applied Bayesian Statistics and Estimation for Social Scientists

Springer, 2007. — 375 р. — (Statistics for Social and Behavioral Sciences). — ISBN 978-0-387-71264-2.Introduction to Bayesian Statistics and Estimation for Social Scientists covers the complete process of Bayesian statistical analysis in great detail from the development of a model through the process of making statistical inference. The key feature of this book is that it covers models that are most commonly used in social science research-including the linear regression model, generalized linear models, hierarchical models, and multivariate regression models-and it thoroughly develops each real-data example in painstaking detail.
Throughout the text, I present R programs for virtually all MCMC algorithms in order to demystify the linkage between model development and estimation. R is a freely available, downloadable programming package and is extremely well suited to Bayesian analyses (www.r-project.org). However, R is only one possible programming language in which MCMC algorithms can be written. Another package I use in the chapter on hierarchical modeling is WinBugs. WinBugs is a freely available, downloadable software package that performs Gibbs sampling with relative ease (www.mrc-bsu.cam.ac.uk/bugs). I strongly suggest learning how to use WinBugs if you expect to routinely conduct Bayesian analyses. The syntax of WinBugs is very similar to R, and so the learning curve is not steep once R is familiar. The key advantage to WinBugs over R is that WinBugs derives conditional distributions for Gibbs sampling for you; the user simply has to specify the model. In R, on the other hand, the conditional distributions must be derived mathematically by the user and then programmed. The key advantage of R over WinBugs, however, is that R—as a generic programming language—affords the user greater flexibility in reading data from files, modeling data, and writing output to files. For learning how to program in R, I recommend downloading the various documentation available when you download the software. I also recommend Venables and Ripley’s books for S and S-Plus R _ programming (1999, 2000). The S and S-Plus languages are virtually identical to R, but they are not freely available.Outline
A note on programming
Symbols used throughout the book
Probability Theory and Classical Statistics
Rules of probability
Probability distributions in general
Some important distributions in social science
Classical statistics in social science
Maximum likelihood estimation
Conclusions
Exercises
Basics of Bayesian Statistics
Bayes’ Theorem for point probabilities
Bayes’ Theorem applied to probability distributions
Bayes’ Theorem with distributions: A voting example
A normal prior–normal likelihood example with σ2 known
Some useful prior distributions
Criticism against Bayesian statistics
Conclusions
Exercises
Modern Model Estimation Part 1: Gibbs Sampling
What Bayesians want and why
The logic of sampling from posterior densities
Two basic sampling methods
Introduction to MCMC sampling
Conclusions
Exercises
Modern Model Estimation Part 2: Metroplis–Hastings Sampling
A generic MH algorithm
Example: MH sampling when conditional densities are difficult to derive
Example: MH sampling for a conditional density with an unknown form
Extending the bivariate normal example: The full multiparameter model
Conclusions
Exercises
Evaluating Markov Chain Monte Carlo Algorithms and Model Fit
Why evaluate MCMC algorithm performance?
Some common problems and solutions
Recognizing poor performance
Evaluating model fit
Formal comparison and combining models
Conclusions
Exercises
The Linear Regression Model
Development of the linear regression model
Sampling from the posterior distribution for the model parameters
Example: Are people in the South “nicer” than others?
Incorporating missing data
Conclusions
Exercises
Generalized Linear Models
The dichotomous probit model
The ordinal probit model
Conclusions
Exercises
Introduction to Hierarchical Models
Hierarchical models in general
Hierarchical linear regression models
A note on fixed versus random effects models and other terminology
Conclusions
Exercises
Introduction to Multivariate Regression Models
Multivariate linear regression
Multivariate probit models
A multivariate probit model for generating distributions
Conclusions
Exercises
Conclusion
Background Mathematics
Summary of calculus
Summary of matrix algebra
Exercises
The Central Limit Theorem, Confidence Intervals, and Hypothesis Tests
A simulation study
Classical inference