1. Reliabilities of the variables. The magnitude and pattern both matter. If all the variables in the matrix have the same reliability, regardless of the magnitude, the pattern of correlations observed, the pattern is easier to interpret.

2. Sampling variance. The smaller the sample size, the more correlations will wander randomly about their population values. The smaller the sample size, the more difficult it is to interpret the observed correlations. This has a special impact on Campbell & Fiske's criteria 2 and 3, that is, the validity diagonal should be larger than relevant entries in the hets and monos.

3. Method variance. The presence and degree of method variance, or method factors, tends to inflate correlations in the monomethod triangles. The more method variance, the more correlations are inflated. You might think of this as correlated error variance between variables that share a common method.

4. Method correlation. Suppose we have method variance and two methods. The method variance will increase the correlations within each heterotrait monomethod triangle (as in point 3). Suppose that the two methods are correlated, so that the method variance is partially common to both methods. Such bias will cause the heterotrait heteromethod coronations to be inflated, ad will also inflate the validity diagonal. Suppose, for example, we are measuring personality with three measures: (1) a self administered personality test, (2) a self-administered graphic rating scale for each personality trait, and (3) peer ratings on a graphic rating scale for each personality trait. What would we expect? The methods most similar are probably (1) and (2) because the source of information in both cases is the self. Any defensiveness attempts to look good to other people will appear in both self report measures. Any common bias between (1) and (2) will result in inflated correlations in the heteromethod block. Methods (2) and (3) also share a common characteristic, the graphic rating scale. Any peculiarities of the scale, such as labels or anchors that influence the ratings will result in correlated method bias. To estimate the magnitude of this bias, we should probably collect other data. We could, for example, collect rankings on the traits from both self and peers. Then we could examine whether and to what degree particular formats (ranking, graphic rating) contribute to correlated method bias. Not the similarity to generalizability theory. We tend to talk about validity when the two variables are maximally different, and reliability when the variables are very similar.

5. Method by trait interaction. Peculiar methods may be better at measuring some traits than other methods. For example, a paper and pencil test will not confuse beauty and intelligence, but a human judge may. The effect off method by trait interaction is that the pattern off correlations will differ across triangles. The monomethod triangles will have one pattern, and the heteromethod triangles will have another.

6. Correlation among traits. Traits may be correlated, and this will raise (properly or correctly) the correlation among all the entries in the matrix, but most importantly the hets will rise as they should. Campbell & Fiske prefer the hets to approach zero. If the traits are uncorrelated, the monomethod triangles should be zero. Correlations larger than zero would be estimates of method bias. If the traits are uncorrelate, the hets should also be zero, and positive correlations indicate method correlation, that is, shared method bias. If we knew the amount of shared method bias, we could reduce the validity diagonal by an appropriate amount and better estimate the amount of convergence.

Usually we do not know the true correlations among the traits, which may be correlated to some degree. Such ignorance introduced ambiguity into the interpretation off the MTMM. If all the entries in the MTMM are large, it is because the traits are correlated, the methods are correlated, or both? It is very difficult to say.

Matrix 1

Simple Perfect MTMM Matrix

1. Reliabilities are equal (.81).

2. Sampling variance is zero (infinite sample size).

3. No method variance.

4. No method correlation.

5. No method by trait interaction.

6. No correlation among the traits.

Matrix 2

Simple Perfect MTMM Matrix with Sampling Error

1. Reliabilities are equal (.81).

2. Sampling variance (N = 100).

3. No method variance.

4. No method correlation.

5. No method by trait interaction.

6. No correlation among the traits.

Matrix 3

Method bias present

1. Reliabilities are equal.

2. Sampling variance zero.

3. Presence of method variance.

4. No method correlation.

5. No method by trait interaction.

6. No correlation among the traits.

Matrix 4

Method bias and method correlation present

1. Reliabilities are equal.

2. Sampling variance zero.

3. Presence of method variance.

4. Presence of method correlation.

5. No method by trait interaction.

6. No correlation among the traits.

Matrix 5

Correlated traits

1. Reliabilities are equal.

2. Sampling variance zero.

3. No method variance.

4. No method correlation.

5. No method by trait interaction.

6. Correlations (.7 and .3) among the traits.

Matrix 6

Correlated traits, reliability varies

1. Reliabilities vary.

2. Sampling variance zero.

3. No method variance.

4. No method correlation.

5. No method by trait interaction.

6. Correlation among the traits.