Principles of assessment of aptitude and achievement, Tutorial

Predicted Achievement Using Simple Linear Regression

There are two ways to make an estimate of a person’s abilities. A point estimate (a single number) is precise but usually wrong, whereas an interval estimate (a range of numbers) is usually right but can be so wide that it is nearly useless. Confidence intervals combine both types of estimates in order to balance the weaknesses of one type of estimate with the strengths of the other. If I say that Suzie’s expected reading comprehension is 85 ± 11, the 85 is the point estimate (also known as the expected score or the predicted score or just Ŷ). The ± 11 is called the margin of error. If the confidence level is left unspecified, by convention we mean the 95% margin of error. If I add 11 and subtract 11 to get a range from 74 to 96, I have the respective lower and upper bounds of the 95% confidence interval.

Calculating the Predicted Achievement Score

I will assume that both the IQ and achievement scores are index scores (μ = 100, σ = 15) to make things simple. The predicted achievement score is a point estimate. It represents the best guess we can make in the absence of other information. The equation below is called a regression equation.

\hat{Y}=\sigma_Y r_{XY} \frac{X-\mu_X}{\sigma_X}+\mu_Y

If X is IQ, Y is Achievement, and both scores are index scores (μ = 100, σ = 15), the regression equation simplifies to:

Predicted achievement = (Correlation between IQ and Achievement) (IQ – 100) + 100

Calculating the Confidence Interval for the Predicted Achievement Score

Whenever you make a prediction using regression, your estimate is not exactly right very often. It is expected to differ from the actual achievement score by a certain amount (on average). This amount is called the standard error of the estimate. It is the standard deviation of all the prediction errors. Thus, it is the standard to which all the errors in your estimates are compared. When both scores are index scores, the formula is

\text{Standard error of the estimate}=\sqrt{1-r^2_{XY}}

To calculate the margin of error, multiply the standard error of the estimate by the z-score that corresponds to the degree of confidence desired. In Microsoft Excel the formula for the z-score corresponding to the 95% confidence interval is

=NORMSINV(1-(1-0.95)/2)

≈1.96

For the 95% confidence interval, multiply the standard error of the estimate by 1.96. The 95% confidence interval’s formula is

95% Confidence Interval = Predicted achievement ± 1.96 * Standard error of the estimate

This interval estimates the achievement score for 95% of people with the same IQ as the child. About 2.5% will score lower than this estimate and 2.5% will score higher.

You can use Excel to estimate how unusual it is for an observed achievement score to differ from a predicted achievement score in a particular direction by using this formula,

=NORMSDIST(-1*ABS(Observed-Predicted)/(Standard error of the estimate))

If a child’s observed achievement score is unusually low, it does not automatically mean that the child has a learning disorder. Many other things need to be checked before that diagnosis can be considered valid. However, it does mean that an explanation for the unusually low achievement score should be sought.

This post is an excerpt from:

Schneider, W. J. (2013). Principles of assessment of aptitude and achievement. In D. Saklofske, C. Reynolds, & V. Schwean (Eds.), Oxford handbook of psychological assessment of children and adolescents (pp. 286–330). New York: Oxford

Standard

2 thoughts on “Predicted Achievement Using Simple Linear Regression”

  1. MKK says:

    What about achievement score that are >1 SD higher than the measured FSIQ? I’m working with a student who has either a) ID, or b) severe language impairment (expressive/receptive) along with visual processing deficits and visual- motor dysfunction. the parents claim the family has a history of language disorders, and assert that the child’s functional skills are on par with peers. The parents disagree with ID as the educational label, and have an independent evaluator who agrees that the child has complex LDs. How do you differentiate?

    FSIQ measures in the low 60s. Achievement scores are generally in the low 80s. Parents report that the student rides their bike to school (.8 mile) and home each day with no problems, participates in team sports, and can pack for a weekend away without adult assistance.

    Reply

    • These things can become counter-intuitive at the extremes. Let’s assume that the correlation of FSIQ and most academic scores is about 0.6. Among people with FSIQ = 61, it is actually typical for academic achievement scores to be 1 SD above the FSIQ. Why?
      Predicted Achievement = 0.6 * (61 – 100) + 100
      Predicted Achievement = 76.6
      Under these assumptions (and multivariate normality), about 39% of people with FSIQ = 61 will have academic achievement scores of 80 or greater. A whopping 13% will have academic achievement scores of 90 or more and 2.6% will score 100 or more.

      From a forthcoming paper:

      How many of us are guilty of thinking that something has gone wrong when we encounter a person whose academic achievement is substantially higher than his or her measured IQ? Such cases are presented as sweet victories for people who hate IQ tests and embarrassments for people who defend them. However, has the IQ test actually failed in such situations? Hardly.

      Once we acknowledge that academic achievement has multiple determinants, such cases are seen to be inevitable. In fact, the child’s low IQ score has revealed something important that might otherwise be overlooked. When a child with a low IQ succeeds (or even performs adequately) in school, it is likely that something special has happened. Perhaps the child has compensating virtues (e.g., conscientiousness) or one or more responsible adults in the child’s life have provided exemplary instruction and care. Far from making a failed prediction, the IQ test has prompted us to investigate what went right in the child’s life when we otherwise might not have noticed that anything was unusual. Such knowledge can be extremely useful for future planning.

      Reply

Leave a comment