4. Evidence for irresolvable differences in how people reason about certain problems

К оглавлению1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 
85 86 87 88 89 90 91 92 93 94 

In Who is Rational (1999), Keith Stanovich sets as his goal to investigate

individual differences across the various HB tasks (as well as other cognitive

tasks). He argues that many views about the HB tradition have

focused too much on subjects’ modal (average) response patterns to the

tasks. As a result, debates about how to properly reason about the tasks are

too often framed in terms of who is wrong—the subjects or the psychologists

(i.e., the proponents of the HB program). That is precisely how

Cohen understands the debate. (But this is not true of Gigerenzer’s frequentist

argument. According to that argument, for tasks asking for singleevent

probabilities, no answer is an error. No psychologists or subjects are

making an error.) The problem with framing the debate in terms of

whether the psychologists or the subjects are wrong is that some subjects

give the response that the psychologists think is the right one. ‘‘Thus, the

issue is not the untutored average person versus experts . . . , but experts

plus some laypersons versus other untutored individuals’’ (61).

There are a number of studies that show that there is considerable

correlation in subjects’ scores on different reasoning tasks associated with

the HB tradition. In other words, subjects who did well on one reasoning

task tended to do well on other tasks. For example, Stanovich and West

(1998) gave subjects four reasoning tasks: syllogistic reasoning, the selection

task, statistical reasoning tasks (pallid statistics vs. vivid single examples),

and argument evaluation (informal reasoning). They found that

‘‘five of the six correlations between these four tasks were significant at the

.001 level’’ (Stanovich 1999, 36). Further, Stanovich and West found that

SAT scores correlate significantly with all four rational thinking tasks.

Interestingly enough, however, for most of the tasks, the subjects who

agreed with the psychologists did not (on average) have more math education

(Stanovich 1999, 40–2).

Another series of studies that might be relevant to properly understanding

the HB literature suggests that when subjects are forced to

articulate justifications for their answers (or otherwise are made accountable),

fewer subjects offer the answer considered wrong by the proponents

of the heuristics and biases tradition (Miller and Fagley 1991, Sieck

and Yates 1997, Takemura 1992, 1993, 1994). On the HB view, this makes

sense—forced to provide a justification, more people get the right answer.

Cohen, and anyone who claims that subjects’ modal answers are correct,

seems forced to claim the opposite—forced to provide a justification,

more people get the wrong answer.

Stanovich reports on a series of studies in which subjects are offered

arguments for and against a particular solution to a task (Stanovich 1999,

81–3). Consider an example of base rate neglect. Subjects were asked to

judge whether the base rate of a fictitious disease (Digirosa) is relevant to

the issue of whether a subject with a red rash has the disease. As expected,

most subjects did not deem the base rates to be relevant. Subjects were

then asked to evaluate one or both of the following arguments.

The percentage of people with Digirosa is needed to determine the probability

because, if Digirosa is very infrequent in the population and some

people without Digirosa also have red rashes, then the probability of

Digirosa might still be low even if the person has a red rash.

The percentage of people with Digirosa is irrelevant because this particular

patient has a red rash, and thus the percentage of people who have Digirosa

is not needed when trying to determine the probability that someone has

Digirosa given that they have a red rash.

The question was: How many subjects would change their mind after being

given one (or both) of these arguments? After being given the argument for

attending to base rates, subjects more often changed their mind in the

‘‘correct’’ (according to the HB program) direction, and this was a statistically

significant difference (39% of those who originally took base rates to

be irrelevant changed their minds after reading the first argument and took

base rates to be relevant, while 15% of those who originally took base rates

to be relevant changed their minds after reading the first argument and

took base rates to be irrelevant). After being given the argument for

neglecting base rates, subjects more often changed their minds in the

‘‘incorrect’’ direction, but this difference was not statistically significant

(15.7% vs 30.8%). After being given both arguments, subjects more often

changed their minds in the ‘‘correct’’ direction, and this was a statistically

significant difference (27.2% vs. 11.1%) (Stanovich 1999, 81–3).

There are three general facts about the studies Stanovich cites that

raise problems for Cohen’s reject-the-norm argument (and for any rejectthe-

norm argument that takes subjects’ modal answers to be correct). The

first is that subjects who tend to reason ‘‘correctly’’ on one problem also

tend to reason ‘‘correctly’’ on other problems (‘‘correctly’’ from the perspective

of the proponents of the HB program). The second fact is that

many subjects’ responses to these tasks are quite malleable:

Collapsed across all of the problems in this chapter that were tested with the

argument evaluation procedure, when presented with an argument on each

side of the question, an average of 22.1% altered their responses on a readministration

of the task. When presented with a single normative argument,

42.5% of the non-normative subjects switched to the normative response on a

readministration of the task. Note that, in the procedure used, subjects were

not told that the argument was correct. They were simply told to evaluate the

argument, and were free to rate it as very weak. . . . Nevertheless, fully 25.4% of

the non-normative subjects shifted after seeing one conflicting argument along

with a compatible one. (Stanovich 1999, 95–6)

And the third fact is the mirror image of this one—lots of people don’t

change their minds on these problems. In fact, most people did not change

their minds even in the face of arguments against their view (including

some quite powerful arguments). What this suggests is that it is quite

likely that subjects will have irreducible disagreements on some of these

problems. (We mean that a significant percentage of subjects who aren’t

explicitly trained to solve these problems a certain way will adopt quite

different solutions.)

If the irreducible disagreement hypothesis is true, what this means is

that Cohen (and anyone who takes subjects’ modal answers to be correct)

will be pushed toward an uncomfortable dilemma. Cohen can insist on

one set of norms—either his or those of the HB program. If he accepts the

norms of the HB program, then he has totally abdicated his position. If he

insists on his own (anti-HB) norms, then he ends up having to argue for

more than he bargained for. It’s not just that the psychologists are wrong;

it’s that the psychologists and lots of subjects are wrong. But this is not a

particularly comfortable position to take. Recall that the subjects who

reason according to the anti-HB norms also tend to do worse on various

cognitive and aptitude tests. While it is not clear what these tests actually

test for, they are correlated with academic success. So if given a choice

between reasoning norms that tend to be followed by better students and

reasoning norms that tend to be followed by worse students, it would be prima facie odd to insist upon the latter. In order to avoid abdication (by

embracing the HB norms) or embarrassment (by embracing the anti-HB

norms), an obvious move is to not insist on just one set of norms. The

proponent of the Cohen-type reject-the-norm strategy might insist that

different norms are right for different reasoners. In this way, Cohen-style

reject-the-norm strategies seem drawn to some kind of ‘‘anything goes’’

epistemic relativism. But this won’t work. Cohen can’t be a relativist. After

all, he argues that the proponents of the HB program are insisting upon

the wrong reasoning rule in assessing the reasoning of their subjects. This

is not consistent with relativism. So those, like Cohen, who advance a

conceptual reject-the-norm argument to defend the view that subjects’

modal answers are correct, are stuck with the abdication-or-embarrassment

dilemma.

In Who is Rational (1999), Keith Stanovich sets as his goal to investigate

individual differences across the various HB tasks (as well as other cognitive

tasks). He argues that many views about the HB tradition have

focused too much on subjects’ modal (average) response patterns to the

tasks. As a result, debates about how to properly reason about the tasks are

too often framed in terms of who is wrong—the subjects or the psychologists

(i.e., the proponents of the HB program). That is precisely how

Cohen understands the debate. (But this is not true of Gigerenzer’s frequentist

argument. According to that argument, for tasks asking for singleevent

probabilities, no answer is an error. No psychologists or subjects are

making an error.) The problem with framing the debate in terms of

whether the psychologists or the subjects are wrong is that some subjects

give the response that the psychologists think is the right one. ‘‘Thus, the

issue is not the untutored average person versus experts . . . , but experts

plus some laypersons versus other untutored individuals’’ (61).

There are a number of studies that show that there is considerable

correlation in subjects’ scores on different reasoning tasks associated with

the HB tradition. In other words, subjects who did well on one reasoning

task tended to do well on other tasks. For example, Stanovich and West

(1998) gave subjects four reasoning tasks: syllogistic reasoning, the selection

task, statistical reasoning tasks (pallid statistics vs. vivid single examples),

and argument evaluation (informal reasoning). They found that

‘‘five of the six correlations between these four tasks were significant at the

.001 level’’ (Stanovich 1999, 36). Further, Stanovich and West found that

SAT scores correlate significantly with all four rational thinking tasks.

Interestingly enough, however, for most of the tasks, the subjects who

agreed with the psychologists did not (on average) have more math education

(Stanovich 1999, 40–2).

Another series of studies that might be relevant to properly understanding

the HB literature suggests that when subjects are forced to

articulate justifications for their answers (or otherwise are made accountable),

fewer subjects offer the answer considered wrong by the proponents

of the heuristics and biases tradition (Miller and Fagley 1991, Sieck

and Yates 1997, Takemura 1992, 1993, 1994). On the HB view, this makes

sense—forced to provide a justification, more people get the right answer.

Cohen, and anyone who claims that subjects’ modal answers are correct,

seems forced to claim the opposite—forced to provide a justification,

more people get the wrong answer.

Stanovich reports on a series of studies in which subjects are offered

arguments for and against a particular solution to a task (Stanovich 1999,

81–3). Consider an example of base rate neglect. Subjects were asked to

judge whether the base rate of a fictitious disease (Digirosa) is relevant to

the issue of whether a subject with a red rash has the disease. As expected,

most subjects did not deem the base rates to be relevant. Subjects were

then asked to evaluate one or both of the following arguments.

The percentage of people with Digirosa is needed to determine the probability

because, if Digirosa is very infrequent in the population and some

people without Digirosa also have red rashes, then the probability of

Digirosa might still be low even if the person has a red rash.

The percentage of people with Digirosa is irrelevant because this particular

patient has a red rash, and thus the percentage of people who have Digirosa

is not needed when trying to determine the probability that someone has

Digirosa given that they have a red rash.

The question was: How many subjects would change their mind after being

given one (or both) of these arguments? After being given the argument for

attending to base rates, subjects more often changed their mind in the

‘‘correct’’ (according to the HB program) direction, and this was a statistically

significant difference (39% of those who originally took base rates to

be irrelevant changed their minds after reading the first argument and took

base rates to be relevant, while 15% of those who originally took base rates

to be relevant changed their minds after reading the first argument and

took base rates to be irrelevant). After being given the argument for

neglecting base rates, subjects more often changed their minds in the

‘‘incorrect’’ direction, but this difference was not statistically significant

(15.7% vs 30.8%). After being given both arguments, subjects more often

changed their minds in the ‘‘correct’’ direction, and this was a statistically

significant difference (27.2% vs. 11.1%) (Stanovich 1999, 81–3).

There are three general facts about the studies Stanovich cites that

raise problems for Cohen’s reject-the-norm argument (and for any rejectthe-

norm argument that takes subjects’ modal answers to be correct). The

first is that subjects who tend to reason ‘‘correctly’’ on one problem also

tend to reason ‘‘correctly’’ on other problems (‘‘correctly’’ from the perspective

of the proponents of the HB program). The second fact is that

many subjects’ responses to these tasks are quite malleable:

Collapsed across all of the problems in this chapter that were tested with the

argument evaluation procedure, when presented with an argument on each

side of the question, an average of 22.1% altered their responses on a readministration

of the task. When presented with a single normative argument,

42.5% of the non-normative subjects switched to the normative response on a

readministration of the task. Note that, in the procedure used, subjects were

not told that the argument was correct. They were simply told to evaluate the

argument, and were free to rate it as very weak. . . . Nevertheless, fully 25.4% of

the non-normative subjects shifted after seeing one conflicting argument along

with a compatible one. (Stanovich 1999, 95–6)

And the third fact is the mirror image of this one—lots of people don’t

change their minds on these problems. In fact, most people did not change

their minds even in the face of arguments against their view (including

some quite powerful arguments). What this suggests is that it is quite

likely that subjects will have irreducible disagreements on some of these

problems. (We mean that a significant percentage of subjects who aren’t

explicitly trained to solve these problems a certain way will adopt quite

different solutions.)

If the irreducible disagreement hypothesis is true, what this means is

that Cohen (and anyone who takes subjects’ modal answers to be correct)

will be pushed toward an uncomfortable dilemma. Cohen can insist on

one set of norms—either his or those of the HB program. If he accepts the

norms of the HB program, then he has totally abdicated his position. If he

insists on his own (anti-HB) norms, then he ends up having to argue for

more than he bargained for. It’s not just that the psychologists are wrong;

it’s that the psychologists and lots of subjects are wrong. But this is not a

particularly comfortable position to take. Recall that the subjects who

reason according to the anti-HB norms also tend to do worse on various

cognitive and aptitude tests. While it is not clear what these tests actually

test for, they are correlated with academic success. So if given a choice

between reasoning norms that tend to be followed by better students and

reasoning norms that tend to be followed by worse students, it would be prima facie odd to insist upon the latter. In order to avoid abdication (by

embracing the HB norms) or embarrassment (by embracing the anti-HB

norms), an obvious move is to not insist on just one set of norms. The

proponent of the Cohen-type reject-the-norm strategy might insist that

different norms are right for different reasoners. In this way, Cohen-style

reject-the-norm strategies seem drawn to some kind of ‘‘anything goes’’

epistemic relativism. But this won’t work. Cohen can’t be a relativist. After

all, he argues that the proponents of the HB program are insisting upon

the wrong reasoning rule in assessing the reasoning of their subjects. This

is not consistent with relativism. So those, like Cohen, who advance a

conceptual reject-the-norm argument to defend the view that subjects’

modal answers are correct, are stuck with the abdication-or-embarrassment

dilemma.