Because cognitive abilities are positively correlated, there is an assumption that cognitive abilities should be evenly developed. When psychologists examine cognitive profiles, they often describe any features that deviate from the expected flat profile.
It is true, mathematically, that the expected profile IS flat. However, this does not mean that flat profiles are common. There is a very large set of possible profiles and only a tiny fraction are perfectly flat. Profiles that are nearly flat are not particularly common, either. Variability is the norm.
Sometimes it helps to get a sense of just how uneven cognitive profiles typically are. That is, it is good to fine-tune our intuitions about the typical profile with many exemplars. Otherwise it is easy to convince ourselves that the reason that we see so many interesting profiles is that we only assess people with interesting problems.
If we use the correlation matrix from the WAIS-IV to randomly simulate multivariate normal profiles, we can see that even in the general population, flat, “plain-vanilla” profiles are relatively rare. There are features that draw the eye in most profiles.
If cognitive abilities were uncorrelated, profiles would be much more uneven than they are. But even with moderately strong positive correlations, there is still room for quite a bit of within-person variability.
Let’s see what happens when we look at profiles that have the exact same Full Scale IQ (80, in this case). The conditional distributions of the remaining scores are seen in the “violin” plots. There is still considerable diversity of profile shape even though the Full Scale IQ is held constant.
Note that the supplemental subtests have wider conditional distributions because they are not included in the Full Scale IQ, not necessarily because they are less g-loaded.
Assessing Psyche, Engaging Gauss, Seeking Sophia 
Dear Dr. Schneider,
I have found it enlighening to read about the low internal consistency reliability of subindexes and subtests, which may make the clinical reading of profiles a rather unsafe undertaking.
Ref: Gignac, G. E., & Watkins, M. W. (2013). Bifactor modeling and the estimation of model-based reliability in the WAIS-IV. Multivariate Behavioral Research, 48, 639-662.
The paper can be downloaded from: http://drgillesgignac.blogspot.dk/p/publications.html?m=0. It’s the fifth pubication from the top.
Best regards
Fróði Debes
Psychologist
Hi Fróði,
Thanks for the link to Gignac’s publications! I am aware of these problems and I take them seriously. I have devoted a good deal of time trying to find ways address these kinds of limitations when I do cognitive assessment.
The paper you cited shows that equally weighted composite scores provide poor estimates of the nested VC, PR, WM and PS latent variables. One reason that this is so is that such scores are highly g-loaded (and the nested latent variables are uncorrelated with g).
As you know, I have advocated a different approach in which latent variables are directly estimated with regression equations. I then put confidence intervals around those estimates. If we were to use a bifactor model, the g-factor would be estimated separately and the estimates of the other factors would be mostly uncontaminated by g. In this manner, we can estimate nested latent variable scores with much higher precision. However, if the reliability of the estimates is still low, then the confidence intervals will be very wide. With very wide confidence intervals, the clinician will know when to be cautious. See this post for more information: http://wp.me/p177GY-li
As a side note, I believe that our focus on thresholds of “acceptable” reliability (e.g., 0.80) is misguided. Not only are such thresholds arbitrary, reliability coefficients fail to convey information in ways that help clinicians make proper decisions. See this post for more on this topic: http://wp.me/p177GY-eP