2.1. The flat maximum principle

Let’s suppose we have an explanation for the success of proper linear

models. It would be natural to suppose we still had a lot of work to do

coming up with an explanation for the success of improper linear models.

But that’s not true. Interestingly enough, it turns out that anyone who

explains the success of proper linear models for problems of human and

social prediction gets for free the explanation of the success of improper

linear models. That’s because for certain kinds of problem, the success of improper models rides piggy-back on the success of proper models. Recall

the passage quoted above in which Dawes reports that many people didn’t

believe his results concerning the success of improper linear models. Here

it is in its entirety:

The results when published engendered two responses. First, many people

didn’t believe them—until they tested out random and unit models on their

own data sets. Then, other people showed that the results were trivial, because

random and unit linear models will yield predictions highly correlated

with those of linear models with optimal weights, and it had already been

shown that optimal linear models outperform global judgments. I concur

with those proclaiming the results trivial, but not realizing their triviality at

the time, I luckily produced a ‘‘citation classic’’—and without being illustrated

with real data sets, the trivial result might never have been so widely

known. (1988, 209, n. 17)

The reason some people argued that Dawes’s results were trivial was because

of a fascinating finding in statistics called the flat maximum principle

(for a good nontechnical explanation, see Lovie and Lovie 1986; for a

more technical introduction, see Einhorn and Hogarth 1975). (Einhorn

and Hogarth in fact show there are not uncommon situations in which the

improper unit weight models will be more reliable than the proper models.

This is in part the result of the overfitting problem; i.e., the proper model

‘‘fits’’ some of the random, unrepresentative peculiarities of the data set on

which it is constructed and is therefore less accurate on future data points

than an improper model.)

The flat maximum principle says that for a certain class of prediction

problems, as long as the signs of the coefficients are right, any linear model

will predict about as well as any other. It is important to recognize that the

flat maximum principle is restricted to certain kinds of problems. In

particular, it applies only to problems in which the following conditions

obtain:

1. The judgment problem has to be difficult. The problem must be such

that no proper model will be especially reliable because the world is

messy. Perhaps the best way to understand this is to visualize it. A linear

model tries to draw a line through a bunch of data points. Suppose the

points are quite spread out so that no single line can get close to all

of them. Two things are intuitively obvious: (a) The best line through

those points won’t be that much better than lots of lines close to it. (b)

The best line through those points might not be the best line through the

next set of spread-out data points that comes down the pike. For example,

consider the attempt to predict what an applicant’s academic

The Amazing Success of Statistical Prediction Rules 33

performance in college might be. Even the best models are not exceptionally

reliable. No one can predict with great accuracy who is and who

is not going to be academically successful in college. A big part of the

reason is colloquially expressed: Stuff happens. Two candidates who are

identical on paper might have quite different academic careers for a

multitude of unpredictable reasons.

2. The evidential cues must be reasonably predictive. The best cues for

predicting academic performance (GPA, test scores) are reasonably predictive.

Certainly, you’ll do better than chance by relying on these cues.

3. The evidential cues must be somewhat redundant. For example, people

with higher GPAs tend to have higher test scores.

Problems of social judgment—who is going to succeed in a job, who is

going to commit another violent act, what football teams are going to win

next weekend—tend to share these features. As a result, for problems of

social judgment, improper models will be about as reliable as proper

models.

Okay, so the success of improper linear models rides piggy-back on

the success of proper linear models for problems of social prediction. So

then what explains the success of proper linear models?

Let’s suppose we have an explanation for the success of proper linear

models. It would be natural to suppose we still had a lot of work to do

coming up with an explanation for the success of improper linear models.

But that’s not true. Interestingly enough, it turns out that anyone who

explains the success of proper linear models for problems of human and

social prediction gets for free the explanation of the success of improper

linear models. That’s because for certain kinds of problem, the success of improper models rides piggy-back on the success of proper models. Recall

the passage quoted above in which Dawes reports that many people didn’t

believe his results concerning the success of improper linear models. Here

it is in its entirety:

The results when published engendered two responses. First, many people

didn’t believe them—until they tested out random and unit models on their

own data sets. Then, other people showed that the results were trivial, because

random and unit linear models will yield predictions highly correlated

with those of linear models with optimal weights, and it had already been

shown that optimal linear models outperform global judgments. I concur

with those proclaiming the results trivial, but not realizing their triviality at

the time, I luckily produced a ‘‘citation classic’’—and without being illustrated

with real data sets, the trivial result might never have been so widely

known. (1988, 209, n. 17)

The reason some people argued that Dawes’s results were trivial was because

of a fascinating finding in statistics called the flat maximum principle

(for a good nontechnical explanation, see Lovie and Lovie 1986; for a

more technical introduction, see Einhorn and Hogarth 1975). (Einhorn

and Hogarth in fact show there are not uncommon situations in which the

improper unit weight models will be more reliable than the proper models.

This is in part the result of the overfitting problem; i.e., the proper model

‘‘fits’’ some of the random, unrepresentative peculiarities of the data set on

which it is constructed and is therefore less accurate on future data points

than an improper model.)

The flat maximum principle says that for a certain class of prediction

problems, as long as the signs of the coefficients are right, any linear model

will predict about as well as any other. It is important to recognize that the

flat maximum principle is restricted to certain kinds of problems. In

particular, it applies only to problems in which the following conditions

obtain:

1. The judgment problem has to be difficult. The problem must be such

that no proper model will be especially reliable because the world is

messy. Perhaps the best way to understand this is to visualize it. A linear

model tries to draw a line through a bunch of data points. Suppose the

points are quite spread out so that no single line can get close to all

of them. Two things are intuitively obvious: (a) The best line through

those points won’t be that much better than lots of lines close to it. (b)

The best line through those points might not be the best line through the

next set of spread-out data points that comes down the pike. For example,

consider the attempt to predict what an applicant’s academic

The Amazing Success of Statistical Prediction Rules 33

performance in college might be. Even the best models are not exceptionally

reliable. No one can predict with great accuracy who is and who

is not going to be academically successful in college. A big part of the

reason is colloquially expressed: Stuff happens. Two candidates who are

identical on paper might have quite different academic careers for a

multitude of unpredictable reasons.

2. The evidential cues must be reasonably predictive. The best cues for

predicting academic performance (GPA, test scores) are reasonably predictive.

Certainly, you’ll do better than chance by relying on these cues.

3. The evidential cues must be somewhat redundant. For example, people

with higher GPAs tend to have higher test scores.

Problems of social judgment—who is going to succeed in a job, who is

going to commit another violent act, what football teams are going to win

next weekend—tend to share these features. As a result, for problems of

social judgment, improper models will be about as reliable as proper

models.

Okay, so the success of improper linear models rides piggy-back on

the success of proper linear models for problems of social prediction. So

then what explains the success of proper linear models?