Shoukri M. Measures of Interobserver Agreement and Reliability

Second Edition. — Taylor and Francis Group, 2011. — 291 p. — e-ISBN: 978-1-4398-1081-1.Measures of Interobserver Agreement and Reliability, Second Edition covers important issues related to the design and analysis of reliability and agreement studies. It examines factors affecting the degree of measurement errors in reliability generalization studies and characteristics influencing the process of diagnosing each subject in a reliability study. The book also illustrates the importance of blinding and random selection of subjects.New to the Second Edition:
1) New chapter that describes various models for methods comparison studies.
2) New chapter on the analysis of reproducibility using the within-subjects coefficient of variation.
3) Emphasis on the definition of the subjects’ and raters’ population as well as sample size determination.This edition continues to offer guidance on how to run sound reliability and agreement studies in clinical settings and other types of investigations. The author explores two ways of producing one pooled estimate of agreement from several centers: a fixed-effect approach and a random sample of centers using a simple meta-analytic approach. The text includes end-of-chapter exercises as well as a CD-ROM of data sets and SAS code.General Introduction.
Review: Measurement Errors in Medical Research, Reliability and Agreement.
Probability and Its Application in Medical Research:
Elements of Probability, Some Rules of Probability, Discrete Random Variables,
Continuous Random Variables, Multivariate Normal Theory and Techniques of Inference,
Parametric Estimation of Population Parameters
(Method of Moments, Moments of Sample Moments, Method of Maximum Likelihood),
Asymptotic Theory of Likelihood Inference.
Applications of Probability in Clinical Epidemiology.
Reliability for Continuous Scale Measurements.
Models for Reliability Studies.
Testing the Equality of Two Independent ICCs.
Testing the Equality of Two Dependent ICCs.
Some Results for the One-Way Random Effects Model when the Data are Unbalanced,
Large Sample Confidence Interval on ρ₂,
Raters Agreement for Continuous Scale Measurements,
Estimating Agreement When Gold Standard Is Present,
Several Faulty Methods and a Gold Standard,
Sample Size Requirements for Design of Reliability Study under One-Way ANOVA,
Sample Size for Case 2.
Estimation of ICC from Two-Way Random Effects Model with Replicated Measurements.
Method-Comparison Studies.
Literature Review, Bland and Altman’s Approach, Bradley–Blackwood Procedure,
Regression Methods for Methods Comparison, Correlated Measurement Errors.
Assessing Agreement in Methods-Comparison Studies with Replicate Measurements.
Population Coefficient of Variation as a Measure of Precision and Reproducibility.
Inference from Single Normal Sample, Coefficient of Variation from Gamma Distribution,
Tests for Equality of Coefficients of Variation,
Statistical Inference on Coefficient of Variation under One-Way Random Effects Model,
Maximum Likelihood Estimation, Variance Stabilizing Transformation.
Estimating WSCV when Error Variance is not Common,
Sample Size Estimation, Analysis of WSCF from Two Dependent Samples:
Likelihood-Based Methodology, Methods for Testing Null Hypothesis, Wald Test, Likelihood Ratio Test, Score Test, Regression Test;
Some Simulation Results.
Measures of Agreement for Dichotomous Outcomes.
Indices of Adjusted Agreement.
Cohen’s Kappa: Chance Corrected Measure of Agreement.
Intraclass Kappa, 2x2 Kappa in Context of Association, Stratified Kappa, Conceptual Issues, Sample Size Requirements,
Dependent Dichotomous Assessments, Adjusting for Covariates.
Simultaneous Assessment of Two Binary Traits by Two Raters,
Coefficients of Agreement for Multiple Rates and Multiple Categories.
Multiple Categories and Two Raters: Category Distinguishability, Test for Interrater Bias;
Agreement for Multiple Raters and Dichotomous Classification:
Exchangeable Ratings, Test for Interrater Bias;
Probability Models:
Bahadur’s Representation, Dirichlet Multinomial Model: Three Categories; Dirichlet Trinomial Model.
Multiple Raters and Multiple Categories.
Testing the Homogeneity of Kappa Statistic from Independent Studies:
Combined Estimator of Kappa from Several Independent Studies: A General Approach,
Fixed Model Approach, Meta-Analysis Approach (Random Effects Model).