1. Robust reliability

Ameliorative Psychology identifies successful reasoning strategies in terms

of their reliability. Goldberg’s Rule is better than clinical judgment, at least

in part, because it is more reliable. But reliability is not enough. It is

important for a reasoning strategy to be robustly reliable—reliable on a

wide range of problems. We noted this in our discussion of the VRAG

(Violence Risk Appraisal Guide) test for predicting violent recidivism (in

chapter 2, section 4.1). The VRAG was originally developed on a group of

Canadian psychiatric patients. But the VRAG is powerful, and can be

recommended, because it is reliable on a much larger set of people. In

other words, the VRAG is robustly reliable.

The importance to Ameliorative Psychology of reasoning strategies of

robust reliability is evident in the objections leveled against certain proposals.

Consider Gigerenzer’s recognition heuristic (Gigerenzer, Todd, and

the ABC Group 1999). Which city has more inhabitants, San Diego or San

Antonio? United States students answered this question correctly 62% of

the time. German students, on the other hand, answered the question correctly

100% of the time (Goldstein and Gigerenzer 1999). Goldstein and

Gigerenzer (1999) took the 22 largest cities in the U.S., randomly paired

them, and asked U.S. students to pick the larger (in terms of inhabitants).

Then they took the 22 largest German cities, randomly paired them, and

asked the students again to pick the larger. The U.S. students did better on

the German cities (median 71% versus median 73%). And when Goldstein

and Gigerenzer ran this same experiment on German students, they found

that the Germans were more accurate on the U.S. cities. They call this the

less-is-more effect : Under certain circumstances, less knowledge can yield

more reliability. What explains the less-is-more effect? Goldstein and

Gigerenzer hypothesize that when subjects are somewhat ignorant about a

subject, it allows them to employ the recognition heuristic : If S recognizes

one of two objects but not the other, and recognition correlates positively

(negatively) with the criterion, then S can infer that the recognized object

has the higher (lower) value. So consider again the San Diego vs. San

Extracting Epistemic Lessons from Ameliorative Psychology 55

Antonio problem. The German students tended to recognize the former

city but not the latter, so they used the recognition heuristic and inferred

(correctly) that San Diego was larger. The U.S. students recognized both

cities and so did not use the recognition heuristic; they made a judgment

on the basis of the knowledge they had about the respective cities. In the

case of San Diego vs. San Antonio, the recognition heuristic was more

reliable.

The obvious worry about the recognition heuristic is that its reliability

depends on whether recognition really does correlate with the target criterion.

This limits the heuristic’s range (i.e., the range of problems on

which it will be reasonably reliable). But it also makes it difficult to discover

the heuristic’s range. (This problem is raised in various ways by

many commentators on Todd and Gigerenzer’s target BBS article [2000].)

The problem can be clearly seen in the application of the recognition

heuristic to the area of investment. Borges, Goldstein, Ortmann, and

Gigerenzer (1999) tested a number of different investment strategies

against the recognition heuristic (investments were selected on the basis of

name recognition). For the six-month period of study, the recognition

heuristic outperformed the other strategies (Borges et al. 1999, 65). Borges

et al. suggest that ordinary people (using the recognition heuristic) can

perhaps do better on the stock market than mutual fund managers and

market indices: ‘‘In investments, there may be wisdom in ignorance’’

(1999, 72). But this conclusion is dubious. In such a short amount of time,

an investment strategy’s results are likely owed to the period of the study

rather than the power of the strategy. Put another way, a six-month period

is unlikely to reliably discriminate between winning and losing strategies.

To perform a valid test of whether the recognition heuristic is better than

other investment strategies, we would need to compare those strategies on

rolling six-month periods over a long period of time (i.e., decades). As

applied to investment, there is reason to suspect that the recognition

heuristic is not robust—it will not reliably identify winners in a wide range

of market environments.

Ameliorative Psychology identifies successful reasoning strategies in terms

of their reliability. Goldberg’s Rule is better than clinical judgment, at least

in part, because it is more reliable. But reliability is not enough. It is

important for a reasoning strategy to be robustly reliable—reliable on a

wide range of problems. We noted this in our discussion of the VRAG

(Violence Risk Appraisal Guide) test for predicting violent recidivism (in

chapter 2, section 4.1). The VRAG was originally developed on a group of

Canadian psychiatric patients. But the VRAG is powerful, and can be

recommended, because it is reliable on a much larger set of people. In

other words, the VRAG is robustly reliable.

The importance to Ameliorative Psychology of reasoning strategies of

robust reliability is evident in the objections leveled against certain proposals.

Consider Gigerenzer’s recognition heuristic (Gigerenzer, Todd, and

the ABC Group 1999). Which city has more inhabitants, San Diego or San

Antonio? United States students answered this question correctly 62% of

the time. German students, on the other hand, answered the question correctly

100% of the time (Goldstein and Gigerenzer 1999). Goldstein and

Gigerenzer (1999) took the 22 largest cities in the U.S., randomly paired

them, and asked U.S. students to pick the larger (in terms of inhabitants).

Then they took the 22 largest German cities, randomly paired them, and

asked the students again to pick the larger. The U.S. students did better on

the German cities (median 71% versus median 73%). And when Goldstein

and Gigerenzer ran this same experiment on German students, they found

that the Germans were more accurate on the U.S. cities. They call this the

less-is-more effect : Under certain circumstances, less knowledge can yield

more reliability. What explains the less-is-more effect? Goldstein and

Gigerenzer hypothesize that when subjects are somewhat ignorant about a

subject, it allows them to employ the recognition heuristic : If S recognizes

one of two objects but not the other, and recognition correlates positively

(negatively) with the criterion, then S can infer that the recognized object

has the higher (lower) value. So consider again the San Diego vs. San

Extracting Epistemic Lessons from Ameliorative Psychology 55

Antonio problem. The German students tended to recognize the former

city but not the latter, so they used the recognition heuristic and inferred

(correctly) that San Diego was larger. The U.S. students recognized both

cities and so did not use the recognition heuristic; they made a judgment

on the basis of the knowledge they had about the respective cities. In the

case of San Diego vs. San Antonio, the recognition heuristic was more

reliable.

The obvious worry about the recognition heuristic is that its reliability

depends on whether recognition really does correlate with the target criterion.

This limits the heuristic’s range (i.e., the range of problems on

which it will be reasonably reliable). But it also makes it difficult to discover

the heuristic’s range. (This problem is raised in various ways by

many commentators on Todd and Gigerenzer’s target BBS article [2000].)

The problem can be clearly seen in the application of the recognition

heuristic to the area of investment. Borges, Goldstein, Ortmann, and

Gigerenzer (1999) tested a number of different investment strategies

against the recognition heuristic (investments were selected on the basis of

name recognition). For the six-month period of study, the recognition

heuristic outperformed the other strategies (Borges et al. 1999, 65). Borges

et al. suggest that ordinary people (using the recognition heuristic) can

perhaps do better on the stock market than mutual fund managers and

market indices: ‘‘In investments, there may be wisdom in ignorance’’

(1999, 72). But this conclusion is dubious. In such a short amount of time,

an investment strategy’s results are likely owed to the period of the study

rather than the power of the strategy. Put another way, a six-month period

is unlikely to reliably discriminate between winning and losing strategies.

To perform a valid test of whether the recognition heuristic is better than

other investment strategies, we would need to compare those strategies on

rolling six-month periods over a long period of time (i.e., decades). As

applied to investment, there is reason to suspect that the recognition

heuristic is not robust—it will not reliably identify winners in a wide range

of market environments.