Meta-Analysis
|
Syllabus |
Homework
|
|
|
Lecture |
Handouts & Supplemental Materials |
|
|
|
Meta-analyst’s Checklist Judgment calls list (Wanous) |
|
1 |
Sampling Distributions, including boxplot & stem-leaf diagrams. |
Background & Meta-analysis method |
|
2 |
Effect Size Definitions |
Effect size, p level, & vote counting. |
|
3 |
Data Collection (beginning a study) Rater reliability & ICCs |
Study & ES retrieval Effect size conversions (move from one type of effect size to another, e.g., d to r) Rater reliability & Intraclass correlations ICC calculator in Excel |
|
4 |
Meta-analysis graphs, i.e., funnel & forest plots. Combining Effect Sizes |
Excel program for this lecture SH SAS program for bare-bones SH SAS Indirect RR for this lecture SH SAS Direct RR
|
|
5 |
Excel sheet for lecture Excel sheet for second example SAS program for bare-bones SAS program for ryy correction
|
|
|
6 |
Fixed-effects vs. Random-effects Larry Hedges’ approach to meta-analysis |
Fixed vs. random Heuristic for model choice Excel sheet for Hedges lecture Some ES formulas and optimal weights.
|
|
7 |
Moderators lecture |
Analog to ANOVA Exel program for effect size r Analog to ANOVA Excel program for d Analog to ANOVA SAS program (Wilson) for effect size r Anolog to ANOVA SAS for d
Weighted regression SAS macro (Wilson) Weighted regression SAS matrix illustration Handout on Moderators SAS program for handout (Lipsey-Wilson weighted regression program) |
|
8 |
Dr. Jeff Kromrey’s methodological research on meta analysis |
Publication Bias Permutation Tests for linear models Estimating the Random Effects Variance Component Robust estimates of Effect size |
|
9 |
Criticisms of Meta-analysis |
|
|
10 |
McNatt illustration- fixed vs. random [write ppt for McNatt; write exercise for Excel] |
McNatt illustration text McNatt illustration spreadsheet |
Meta-analysis Example Programs
This page leads to examples of each type of meta-analysis calculated either using Microsoft Excel or SAS for the computations. Generally speaking, both the Hedges and Hunter/Schmidt methods produce similar overall estimates of the random-effects mean and random-effects variance component (although the estimation techniques are different the actual numbers may be different). The Hedges approach can also be used for fixed-effects meta-analysis, but I generally avoid overall fixed-effects models for meta-analysis. The Hedges approach is commonly preferred for the analysis of moderators because it yields statistical tests of the significance of the moderator effects, and statistical tests of the residual heterogeneity (estimated variance of infinite-sample studies after accounting for the moderator) helpful for deciding on fixed versus mixed models. The Hunter-Schmidt methods are generally preferred for analyses in which multiple artifact adjustments (adjustments for study unreliability, range restriction) are desired.
|
|
|
|
Moderators |
|
|
Hedges and company |
Fixed overall |
Random overall |
Categorical Moderator |
Continuous Moderator |
|
Correlation (r) |
||||
|
Mean diff (d) |
SAS |
|||
|
|
|
|
|
|
|
Hunter & Schmidt |
Fixed Overall |
Random overall ‘bare bones’ |
Correction for reliability |
Multiple correction (direct & indirect range restriction) |
|
Correlation(r) |
NA |
Excel SAS |
Excel SAS indirectRR, SAS directRR |
|
|
Mean diff (d) |
NA |
Excel SAS |
Excel SAS |
Excel SAS |
|
|
|
|
|
|
Other
Programs |
|
|
|
|
|
|
|
|
Filedrawer |
Filedrawer for t |
|
Graphs |
|
Critiques |
Presentations |
|
Reliability Generalization Powerpoint, and Paper |
|
|
Critique of Big5 and Satisfaction |
Organizational Citizenship Behavior |
|
Critique of effects of design on reliability of rating scales in marketing |
Perceived Supervisory Support |
|
Critique of sales satisfaction meta-analysis |
|
|
|
|
|
|
|
|
|
|
|
Thin images and body satisfaction |
|
|
Relapse prevention in substance abuse |
|
|
Counterproductive work behavior |
|
|
Effects of feedback on task performance |
|
|
Short term psychotherapy |
|
|
Sex differences in spatial ability |
|
|
|
|
|
Critique of Anger Treatment MA |
|