McLachlan G., Peel D. Finite Mixture Models

John Wiley, 2000. — 439 p.The importance of finite mixture models in the statistical analysis of data is underscored by the ever-increasing rate at which articles on mixture applications appear in the statistical and general scientific literature. The aim ofthis monograph is to provide an up-to-date account of the theory and applications of modeling via finite mixture distributions. Since the appearance of the monograph of McLachlan and Basford (1988) on finite mixtures, the literature has expanded enormously to the extent that another monograph on the topic is apt. In the past decade the extent and the potential of the applications of finite mixture models have widened considerably. Because of their flexibility, mixture models are being increasingly exploited as a convenient, semi parametric way in which to model unknown distributional shapes. This is in addition to their obvious applications where there is group-structure in the data or where the aim is to explore the data for such structure, as in a cluster analysis.
In this book the more recent work is surveyed against the background of the existing literature. The widespread use of mixture models in recent times is demonstrated by the fact that of the 800 or so references in this book, almost 40% of them have been published since 1995. A comprehensive account of the major issues involved with modeling via finite mixture distributions is provided. They include identifiability problems, the actual fitting of finite mixtures through use of the EM algorithm, the properties of the maximum likelihood estimators so obtained, the assessment of the number of components to be used in the mixture. and the applicability of asymptotic theory in providing a basis for the solutions to some ofthese problems. The intent is to provide guidelines to users of mixture models on these various issues. The emphasis is on the applications of mixture models, not only in mainstream statistical analyses. but also in other areas such as unsupervised pattern recognition, speech recognition, and medical imaging.
With the advent of inexpensive, high-speed computers and the simultaneous rapid development in posterior simulation techniques such as Markov chain Monte Carlo (MCMC) methods for enabling Bayesian estimation to be undertaken, practitioners are increasingly turning to Bayesian methods for the analysis of complicated statistical models. In this book, we consider the latest developments in Bayesian estimation of mixture models.
New topics that are covered in this book include the scaling of the EM algorithm to allow mixture models to be used in data mining applications involving massively huge databases. In the same spirit, there is also an account of the use of the sparse/incremental EM algorithm and of multiresolution kd-trees for speeding up the implementation of the standard EM algorithm for the fitting of mixture models. Another topic concerns the use of hierarchical mixtures-of-experts models as a powerful new approach to nonlinear regression that is a serious competitor to well-known statistical procedures such as MARS and CART. Other recent developments covered include the use of mixture models for handling overdispersion in generalized linear models and proposals for dealing with mixed continuous and categorical variables. In other recent work, there is the proposal to use t components in the mixture model to provide a robust approach to mixture modeling. A further topic is the use of mixtures of factor analyzers which provide a way of fitting mixture models to high-dimensional data. As mixture models provide a convenient basis for the modeling of dependent data by allowing the component-indicator variables to have a Markovian structure, there is also coverage of the latest developments in hidden Markov models, including the Bayesian approach to this problem. Another problem considered is the fitting of mixture models to multivariate data in binned form, which arises in some important medical applications in practice.
The book also covers the latest developments on existing issues with mixture modeling, such as assessing the number of components to be used in a mixture model and the associated problem of determining how many clusters there are in clustering applications with mixture models.
In presenting these latest results, the authors have attempted to draw together the statistical literature with the machine learning and pattern recognition literature. It is intended that the book should appeal to both applied and theoretical statisticians, as well as to investigators working in the many diverse areas in which relevant use can be made of finite mixture models. It will be assumed that the reader has a fair mathematical or statistical background. The main parts of the book describing the formulation of the finite mixture approach, detailing its methodology, discussing aspects of its implementation, and illustrating its application in many simple statistical contexts should be comprehensible to graduates with statistics as their major subject. The emphasis is on the practical applications of mixture models; and to this end, numerous examples are given.General Introduction.
ML Fitting of Mixture Models.
Multivariate Normal Mixtures.
Bayesian Approach to Mixture Analysis.
Mixtures with Nonnormal Components.
Assessing the Number of Components in Mixture Models.
Multivariate t Mixtures.
Mixtures of Factor Analyzers.
Fitting Mixture Models to Binned Data.
Mixture Models for Failure-Time Data.
Mixture Analysis of Directional Data.
Variants of the EM Algorithm for Large Databases.
Hidden Markov Models.
Mixture Software.