1. Conceptual reject-the-norm arguments

Conceptual reject-the-norm arguments cast a very wide net by appealing

to abstract, conceptual considerations to show that subjects are not reasoning

poorly about a large set of problems. It must be clear to the reader

where we come down on conceptual reject-the-norm arguments. According

to Strategic Reliabilism, the relative quality of a reasoning strategy is a

function of its expected costs and benefits as well as those of its competitors.

It is an empirical question which of two reasoning strategies has a better

cost-benefit ratio for a particular reasoner. So Strategic Reliabilism is

committed to the view that optimism about our cognitive abilities must be

held, if at all, for empirical reasons. From the perspective of Strategic

Reliabilism, when the optimist assesses someone’s reasoning, he must

present evidence that these individuals have availed themselves of the best

strategies.

We will focus on two conceptual reject-the-norm arguments that have

received considerable attention, one from a philosopher, another from a

psychologist. Perhaps the most famous (or infamous) conceptual rejectthe-

norm argument was put forward by the philosopher L. J. Cohen. He

argued that it is impossible to empirically demonstrate human irrationality

(1981). More recently, Gerd Gigerenzer has proposed an argument

that is meant to show that subjects’ answers to many of the HB problemtasks

are not really errors. Neither of these arguments focuses on how a

reasoner interprets or approaches or even answers a particular problem;

nor do they attend to the consequences of their reasoning. Gigerenzer and

Cohen both attempt to commit us to a conception of rationality—or at

any rate a conception of cognitive permission or approval—that precedes

evidence. This strategy may satisfy a certain taste for a priori standards

of rationality, but it also provides reasoning advice that can lead to relatively

inferior outcomes. We would warn that when epistemologists consistently

attribute positive normative status to reasoning strategies that predictably and robustly lead to inferior outcomes, the Aristotelian Principle

suggests that something has gone deeply wrong.

The considerations raised by the conceptual reject-the-norm arguments

tend to be highly abstract. So it will be useful to keep our discussion

grounded in an example. We will focus on base rate neglect (since this is

an example both Cohen and Gigerenzer discuss in their work). Base rate

neglect typically occurs when people are trying to infer from symptoms to

causes. John fails an exam, so he must not have studied; Julie tests positive

for drugs, so she must take drugs; Kareem has a cough and a fever, so he

must have the flu. How reliable are these conclusions? The standard way to

approach such problems is to invoke Bayes’ Rule:

P(C=S) ј P(S=C) _ P(C) = _[P(S=C) _ P(C)] ю [P(S=_C) _ P(_C)]_

where C is the alleged cause (not studying, taking drugs, having the flu)

and S is the symptom or effect (failing the exam, failing the drug test,

having a cough and a fever). (Note that Bayes’ Rule may be applied even if

there is no causal connection between C and S.) The probability of C given

S depends essentially on the base rates—on the prior probability of C and

so the prior probability of _C. And yet in a very wide array of situations,

even very sophisticated subjects ignore base rates. Consider an example

presented to sixty students and staff at Harvard Medical School (Casscells,

Schoenberger, Grayboys 1978):

If a test to detect a disease whose prevalence is 1/1,000 has a false positive

rate of 5%, what is the chance that a person found to have a positive result

actually has the disease, assuming you know nothing about the person’s

symptoms or signs?

Casscells et al. did not describe the diagnosis problem fully enough for

someone to properly solve it without making some important assumptions.

In particular, one must assume something about the test’s sensitivity

(i.e., the probability that a person would test positive given that she had

the disease). Casscells et al. assumed that the test’s sensitivity is about

100%. Given this assumption, Bayes’ Rule yields the result that the probability

that the subject has the disease given the positive result is about 2%.

Among the faculty and staff at Harvard, almost half judged the probability

to be 95%; the mean answer was 56%; and only 18% of subjects responded

with the answer given by Bayes’ Rule. (Despite problems with the Casscells

et al. study, their finding has been replicated in studies that include all the

necessary information [Gigerenzer 1996a; Gigerenzer and Hoffrage 1995].)

A reject-the-norm argument will conclude that subjects who do not

give the Bayesian answer to this problem have not violated any epistemological

norms. Their reasoning was neither flawed nor irrational, and

their answers were not errors. Conceptual versions of the reject-the-norm

argument do not depend on contingent, empirical facts about the reasoners.

Let’s turn first to Gigerenzer’s argument.

Conceptual reject-the-norm arguments cast a very wide net by appealing

to abstract, conceptual considerations to show that subjects are not reasoning

poorly about a large set of problems. It must be clear to the reader

where we come down on conceptual reject-the-norm arguments. According

to Strategic Reliabilism, the relative quality of a reasoning strategy is a

function of its expected costs and benefits as well as those of its competitors.

It is an empirical question which of two reasoning strategies has a better

cost-benefit ratio for a particular reasoner. So Strategic Reliabilism is

committed to the view that optimism about our cognitive abilities must be

held, if at all, for empirical reasons. From the perspective of Strategic

Reliabilism, when the optimist assesses someone’s reasoning, he must

present evidence that these individuals have availed themselves of the best

strategies.

We will focus on two conceptual reject-the-norm arguments that have

received considerable attention, one from a philosopher, another from a

psychologist. Perhaps the most famous (or infamous) conceptual rejectthe-

norm argument was put forward by the philosopher L. J. Cohen. He

argued that it is impossible to empirically demonstrate human irrationality

(1981). More recently, Gerd Gigerenzer has proposed an argument

that is meant to show that subjects’ answers to many of the HB problemtasks

are not really errors. Neither of these arguments focuses on how a

reasoner interprets or approaches or even answers a particular problem;

nor do they attend to the consequences of their reasoning. Gigerenzer and

Cohen both attempt to commit us to a conception of rationality—or at

any rate a conception of cognitive permission or approval—that precedes

evidence. This strategy may satisfy a certain taste for a priori standards

of rationality, but it also provides reasoning advice that can lead to relatively

inferior outcomes. We would warn that when epistemologists consistently

attribute positive normative status to reasoning strategies that predictably and robustly lead to inferior outcomes, the Aristotelian Principle

suggests that something has gone deeply wrong.

The considerations raised by the conceptual reject-the-norm arguments

tend to be highly abstract. So it will be useful to keep our discussion

grounded in an example. We will focus on base rate neglect (since this is

an example both Cohen and Gigerenzer discuss in their work). Base rate

neglect typically occurs when people are trying to infer from symptoms to

causes. John fails an exam, so he must not have studied; Julie tests positive

for drugs, so she must take drugs; Kareem has a cough and a fever, so he

must have the flu. How reliable are these conclusions? The standard way to

approach such problems is to invoke Bayes’ Rule:

P(C=S) ј P(S=C) _ P(C) = _[P(S=C) _ P(C)] ю [P(S=_C) _ P(_C)]_

where C is the alleged cause (not studying, taking drugs, having the flu)

and S is the symptom or effect (failing the exam, failing the drug test,

having a cough and a fever). (Note that Bayes’ Rule may be applied even if

there is no causal connection between C and S.) The probability of C given

S depends essentially on the base rates—on the prior probability of C and

so the prior probability of _C. And yet in a very wide array of situations,

even very sophisticated subjects ignore base rates. Consider an example

presented to sixty students and staff at Harvard Medical School (Casscells,

Schoenberger, Grayboys 1978):

If a test to detect a disease whose prevalence is 1/1,000 has a false positive

rate of 5%, what is the chance that a person found to have a positive result

actually has the disease, assuming you know nothing about the person’s

symptoms or signs?

Casscells et al. did not describe the diagnosis problem fully enough for

someone to properly solve it without making some important assumptions.

In particular, one must assume something about the test’s sensitivity

(i.e., the probability that a person would test positive given that she had

the disease). Casscells et al. assumed that the test’s sensitivity is about

100%. Given this assumption, Bayes’ Rule yields the result that the probability

that the subject has the disease given the positive result is about 2%.

Among the faculty and staff at Harvard, almost half judged the probability

to be 95%; the mean answer was 56%; and only 18% of subjects responded

with the answer given by Bayes’ Rule. (Despite problems with the Casscells

et al. study, their finding has been replicated in studies that include all the

necessary information [Gigerenzer 1996a; Gigerenzer and Hoffrage 1995].)

A reject-the-norm argument will conclude that subjects who do not

give the Bayesian answer to this problem have not violated any epistemological

norms. Their reasoning was neither flawed nor irrational, and

their answers were not errors. Conceptual versions of the reject-the-norm

argument do not depend on contingent, empirical facts about the reasoners.

Let’s turn first to Gigerenzer’s argument.