4. Evidence for irresolvable differences in how people reason about certain problems
К оглавлению1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1617 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94
In Who is Rational (1999), Keith Stanovich sets as his goal to investigate
individual differences across the various HB tasks (as well as other cognitive
tasks). He argues that many views about the HB tradition have
focused too much on subjects’ modal (average) response patterns to the
tasks. As a result, debates about how to properly reason about the tasks are
too often framed in terms of who is wrong—the subjects or the psychologists
(i.e., the proponents of the HB program). That is precisely how
Cohen understands the debate. (But this is not true of Gigerenzer’s frequentist
argument. According to that argument, for tasks asking for singleevent
probabilities, no answer is an error. No psychologists or subjects are
making an error.) The problem with framing the debate in terms of
whether the psychologists or the subjects are wrong is that some subjects
give the response that the psychologists think is the right one. ‘‘Thus, the
issue is not the untutored average person versus experts . . . , but experts
plus some laypersons versus other untutored individuals’’ (61).
There are a number of studies that show that there is considerable
correlation in subjects’ scores on different reasoning tasks associated with
the HB tradition. In other words, subjects who did well on one reasoning
task tended to do well on other tasks. For example, Stanovich and West
(1998) gave subjects four reasoning tasks: syllogistic reasoning, the selection
task, statistical reasoning tasks (pallid statistics vs. vivid single examples),
and argument evaluation (informal reasoning). They found that
‘‘five of the six correlations between these four tasks were significant at the
.001 level’’ (Stanovich 1999, 36). Further, Stanovich and West found that
SAT scores correlate significantly with all four rational thinking tasks.
Interestingly enough, however, for most of the tasks, the subjects who
agreed with the psychologists did not (on average) have more math education
(Stanovich 1999, 40–2).
Another series of studies that might be relevant to properly understanding
the HB literature suggests that when subjects are forced to
articulate justifications for their answers (or otherwise are made accountable),
fewer subjects offer the answer considered wrong by the proponents
of the heuristics and biases tradition (Miller and Fagley 1991, Sieck
and Yates 1997, Takemura 1992, 1993, 1994). On the HB view, this makes
sense—forced to provide a justification, more people get the right answer.
Cohen, and anyone who claims that subjects’ modal answers are correct,
seems forced to claim the opposite—forced to provide a justification,
more people get the wrong answer.
Stanovich reports on a series of studies in which subjects are offered
arguments for and against a particular solution to a task (Stanovich 1999,
81–3). Consider an example of base rate neglect. Subjects were asked to
judge whether the base rate of a fictitious disease (Digirosa) is relevant to
the issue of whether a subject with a red rash has the disease. As expected,
most subjects did not deem the base rates to be relevant. Subjects were
then asked to evaluate one or both of the following arguments.
The percentage of people with Digirosa is needed to determine the probability
because, if Digirosa is very infrequent in the population and some
people without Digirosa also have red rashes, then the probability of
Digirosa might still be low even if the person has a red rash.
The percentage of people with Digirosa is irrelevant because this particular
patient has a red rash, and thus the percentage of people who have Digirosa
is not needed when trying to determine the probability that someone has
Digirosa given that they have a red rash.
The question was: How many subjects would change their mind after being
given one (or both) of these arguments? After being given the argument for
attending to base rates, subjects more often changed their mind in the
‘‘correct’’ (according to the HB program) direction, and this was a statistically
significant difference (39% of those who originally took base rates to
be irrelevant changed their minds after reading the first argument and took
base rates to be relevant, while 15% of those who originally took base rates
to be relevant changed their minds after reading the first argument and
took base rates to be irrelevant). After being given the argument for
neglecting base rates, subjects more often changed their minds in the
‘‘incorrect’’ direction, but this difference was not statistically significant
(15.7% vs 30.8%). After being given both arguments, subjects more often
changed their minds in the ‘‘correct’’ direction, and this was a statistically
significant difference (27.2% vs. 11.1%) (Stanovich 1999, 81–3).
There are three general facts about the studies Stanovich cites that
raise problems for Cohen’s reject-the-norm argument (and for any rejectthe-
norm argument that takes subjects’ modal answers to be correct). The
first is that subjects who tend to reason ‘‘correctly’’ on one problem also
tend to reason ‘‘correctly’’ on other problems (‘‘correctly’’ from the perspective
of the proponents of the HB program). The second fact is that
many subjects’ responses to these tasks are quite malleable:
Collapsed across all of the problems in this chapter that were tested with the
argument evaluation procedure, when presented with an argument on each
side of the question, an average of 22.1% altered their responses on a readministration
of the task. When presented with a single normative argument,
42.5% of the non-normative subjects switched to the normative response on a
readministration of the task. Note that, in the procedure used, subjects were
not told that the argument was correct. They were simply told to evaluate the
argument, and were free to rate it as very weak. . . . Nevertheless, fully 25.4% of
the non-normative subjects shifted after seeing one conflicting argument along
with a compatible one. (Stanovich 1999, 95–6)
And the third fact is the mirror image of this one—lots of people don’t
change their minds on these problems. In fact, most people did not change
their minds even in the face of arguments against their view (including
some quite powerful arguments). What this suggests is that it is quite
likely that subjects will have irreducible disagreements on some of these
problems. (We mean that a significant percentage of subjects who aren’t
explicitly trained to solve these problems a certain way will adopt quite
different solutions.)
If the irreducible disagreement hypothesis is true, what this means is
that Cohen (and anyone who takes subjects’ modal answers to be correct)
will be pushed toward an uncomfortable dilemma. Cohen can insist on
one set of norms—either his or those of the HB program. If he accepts the
norms of the HB program, then he has totally abdicated his position. If he
insists on his own (anti-HB) norms, then he ends up having to argue for
more than he bargained for. It’s not just that the psychologists are wrong;
it’s that the psychologists and lots of subjects are wrong. But this is not a
particularly comfortable position to take. Recall that the subjects who
reason according to the anti-HB norms also tend to do worse on various
cognitive and aptitude tests. While it is not clear what these tests actually
test for, they are correlated with academic success. So if given a choice
between reasoning norms that tend to be followed by better students and
reasoning norms that tend to be followed by worse students, it would be prima facie odd to insist upon the latter. In order to avoid abdication (by
embracing the HB norms) or embarrassment (by embracing the anti-HB
norms), an obvious move is to not insist on just one set of norms. The
proponent of the Cohen-type reject-the-norm strategy might insist that
different norms are right for different reasoners. In this way, Cohen-style
reject-the-norm strategies seem drawn to some kind of ‘‘anything goes’’
epistemic relativism. But this won’t work. Cohen can’t be a relativist. After
all, he argues that the proponents of the HB program are insisting upon
the wrong reasoning rule in assessing the reasoning of their subjects. This
is not consistent with relativism. So those, like Cohen, who advance a
conceptual reject-the-norm argument to defend the view that subjects’
modal answers are correct, are stuck with the abdication-or-embarrassment
dilemma.
In Who is Rational (1999), Keith Stanovich sets as his goal to investigate
individual differences across the various HB tasks (as well as other cognitive
tasks). He argues that many views about the HB tradition have
focused too much on subjects’ modal (average) response patterns to the
tasks. As a result, debates about how to properly reason about the tasks are
too often framed in terms of who is wrong—the subjects or the psychologists
(i.e., the proponents of the HB program). That is precisely how
Cohen understands the debate. (But this is not true of Gigerenzer’s frequentist
argument. According to that argument, for tasks asking for singleevent
probabilities, no answer is an error. No psychologists or subjects are
making an error.) The problem with framing the debate in terms of
whether the psychologists or the subjects are wrong is that some subjects
give the response that the psychologists think is the right one. ‘‘Thus, the
issue is not the untutored average person versus experts . . . , but experts
plus some laypersons versus other untutored individuals’’ (61).
There are a number of studies that show that there is considerable
correlation in subjects’ scores on different reasoning tasks associated with
the HB tradition. In other words, subjects who did well on one reasoning
task tended to do well on other tasks. For example, Stanovich and West
(1998) gave subjects four reasoning tasks: syllogistic reasoning, the selection
task, statistical reasoning tasks (pallid statistics vs. vivid single examples),
and argument evaluation (informal reasoning). They found that
‘‘five of the six correlations between these four tasks were significant at the
.001 level’’ (Stanovich 1999, 36). Further, Stanovich and West found that
SAT scores correlate significantly with all four rational thinking tasks.
Interestingly enough, however, for most of the tasks, the subjects who
agreed with the psychologists did not (on average) have more math education
(Stanovich 1999, 40–2).
Another series of studies that might be relevant to properly understanding
the HB literature suggests that when subjects are forced to
articulate justifications for their answers (or otherwise are made accountable),
fewer subjects offer the answer considered wrong by the proponents
of the heuristics and biases tradition (Miller and Fagley 1991, Sieck
and Yates 1997, Takemura 1992, 1993, 1994). On the HB view, this makes
sense—forced to provide a justification, more people get the right answer.
Cohen, and anyone who claims that subjects’ modal answers are correct,
seems forced to claim the opposite—forced to provide a justification,
more people get the wrong answer.
Stanovich reports on a series of studies in which subjects are offered
arguments for and against a particular solution to a task (Stanovich 1999,
81–3). Consider an example of base rate neglect. Subjects were asked to
judge whether the base rate of a fictitious disease (Digirosa) is relevant to
the issue of whether a subject with a red rash has the disease. As expected,
most subjects did not deem the base rates to be relevant. Subjects were
then asked to evaluate one or both of the following arguments.
The percentage of people with Digirosa is needed to determine the probability
because, if Digirosa is very infrequent in the population and some
people without Digirosa also have red rashes, then the probability of
Digirosa might still be low even if the person has a red rash.
The percentage of people with Digirosa is irrelevant because this particular
patient has a red rash, and thus the percentage of people who have Digirosa
is not needed when trying to determine the probability that someone has
Digirosa given that they have a red rash.
The question was: How many subjects would change their mind after being
given one (or both) of these arguments? After being given the argument for
attending to base rates, subjects more often changed their mind in the
‘‘correct’’ (according to the HB program) direction, and this was a statistically
significant difference (39% of those who originally took base rates to
be irrelevant changed their minds after reading the first argument and took
base rates to be relevant, while 15% of those who originally took base rates
to be relevant changed their minds after reading the first argument and
took base rates to be irrelevant). After being given the argument for
neglecting base rates, subjects more often changed their minds in the
‘‘incorrect’’ direction, but this difference was not statistically significant
(15.7% vs 30.8%). After being given both arguments, subjects more often
changed their minds in the ‘‘correct’’ direction, and this was a statistically
significant difference (27.2% vs. 11.1%) (Stanovich 1999, 81–3).
There are three general facts about the studies Stanovich cites that
raise problems for Cohen’s reject-the-norm argument (and for any rejectthe-
norm argument that takes subjects’ modal answers to be correct). The
first is that subjects who tend to reason ‘‘correctly’’ on one problem also
tend to reason ‘‘correctly’’ on other problems (‘‘correctly’’ from the perspective
of the proponents of the HB program). The second fact is that
many subjects’ responses to these tasks are quite malleable:
Collapsed across all of the problems in this chapter that were tested with the
argument evaluation procedure, when presented with an argument on each
side of the question, an average of 22.1% altered their responses on a readministration
of the task. When presented with a single normative argument,
42.5% of the non-normative subjects switched to the normative response on a
readministration of the task. Note that, in the procedure used, subjects were
not told that the argument was correct. They were simply told to evaluate the
argument, and were free to rate it as very weak. . . . Nevertheless, fully 25.4% of
the non-normative subjects shifted after seeing one conflicting argument along
with a compatible one. (Stanovich 1999, 95–6)
And the third fact is the mirror image of this one—lots of people don’t
change their minds on these problems. In fact, most people did not change
their minds even in the face of arguments against their view (including
some quite powerful arguments). What this suggests is that it is quite
likely that subjects will have irreducible disagreements on some of these
problems. (We mean that a significant percentage of subjects who aren’t
explicitly trained to solve these problems a certain way will adopt quite
different solutions.)
If the irreducible disagreement hypothesis is true, what this means is
that Cohen (and anyone who takes subjects’ modal answers to be correct)
will be pushed toward an uncomfortable dilemma. Cohen can insist on
one set of norms—either his or those of the HB program. If he accepts the
norms of the HB program, then he has totally abdicated his position. If he
insists on his own (anti-HB) norms, then he ends up having to argue for
more than he bargained for. It’s not just that the psychologists are wrong;
it’s that the psychologists and lots of subjects are wrong. But this is not a
particularly comfortable position to take. Recall that the subjects who
reason according to the anti-HB norms also tend to do worse on various
cognitive and aptitude tests. While it is not clear what these tests actually
test for, they are correlated with academic success. So if given a choice
between reasoning norms that tend to be followed by better students and
reasoning norms that tend to be followed by worse students, it would be prima facie odd to insist upon the latter. In order to avoid abdication (by
embracing the HB norms) or embarrassment (by embracing the anti-HB
norms), an obvious move is to not insist on just one set of norms. The
proponent of the Cohen-type reject-the-norm strategy might insist that
different norms are right for different reasoners. In this way, Cohen-style
reject-the-norm strategies seem drawn to some kind of ‘‘anything goes’’
epistemic relativism. But this won’t work. Cohen can’t be a relativist. After
all, he argues that the proponents of the HB program are insisting upon
the wrong reasoning rule in assessing the reasoning of their subjects. This
is not consistent with relativism. So those, like Cohen, who advance a
conceptual reject-the-norm argument to defend the view that subjects’
modal answers are correct, are stuck with the abdication-or-embarrassment
dilemma.