Multi Trait Multi Method Modeling of Method Correlations

A construct validity measurement technique introduced in 1959 by Donald Campbell and Donald Fiske, multi trait multi method evaluates the extent to which a measure is influenced by the intended traits versus systematic factors. These systematic factors are often referred to as method effects. One of the main problems with assessing construct validity with multitrait-multimethod designs is the presence of correlations between measures of the same trait measured with different methods. The resulting matrix of correlations is known as the multitrait-multimethod matrix (MTMM), and it is typically used to evaluate convergent and discriminant validity.

To correctly evaluate a MTMM, three criteria are typically used. First, the same-trait, different-method correlations should be larger than the correlations between measures of different traits measured with the same method. This criterion is generally met in Table 1. Second, the same-trait, different-method -correlations on the validity diagonal should be higher than the different-trait, same-method -correlations on that diagonal. This criterion is also generally met in Table 1.

Third, the model should decompose variance into trait and method components. This is done by examining the standardized versions of the model parameters (b0m*, b1m*, and V ar(zm*)). This enables us to see how much of the correlation between the same trait measured with different methods can be explained by the common trait factor T, and how much of it is due to method differences.

It is common to find that the shared trait variance explains very little of the correlation between a measure and another measure. Hence, there is a need to introduce some method variance in the model. Rather than modeling it as an error term, it is commonly modeled as a latent variable that represents the amount of method variance that has to be added to the measured trait (i.e., to get the true correlation). In this article we examine how to incorporate method variance into a multi trait multi method model using the Mplus statistical software.

We will use a data set containing multiple raters’ reports of children’s ADHD inattention symptoms to demonstrate the estimation and interpretation of non-linear trait-method relationships. In particular, we will show how to estimate and interpret quadratic LM and LD models with and without measurement error, as well as a fully specified multitrait-multimethod model. The Mplus syntax for all models is provided in Appendix C.

For the MTMM models, we will start with the assumption that there are t traits and m methods. We will then order the methods based on how fast they move, such that the fastest moving method is first. We will then create m standardized latent variables, denoted as K factors, for the t traits. The loadings for the K factors will be a mixture of trait and method variance, with each measuring one of the t traits. Then we will calculate the correlations between the m measured traits and the K factors, and fix the loadings for the first K factor to be equal for all the methods.