Preface

This book began in our classrooms. At some point, we discovered that

we both teach critical thinking courses that are idiosyncratic in the

same ways—in short, as though they are courses in the psychology of

judgment. For example, we both had our students read Robyn Dawes’s

House of Cards (1994) and Thomas Gilovich’s How We Know What Isn’t So

(1991). We independently arrived at the idea that there were epistemological

lessons to be drawn not just from the heuristics and biases tradition

(which has received attention from philosophers) but also from the fascinating

research on linear predictive modeling. But we also recognized

that psychologists for too long had been wrestling with normative, epistemic

issues with much too little useful input from philosophers. In their

classic book Human Inference: Strategies and Shortcomings of Social Judgment

(1980), Richard Nisbett and Lee Ross call for greater participation

from philosophers in tackling the normative issues that arise in psychology.

[W]e have become increasingly aware of the difficulty of defining what is

‘‘normative’’ when one moves beyond the relatively simple question of how

to solve correctly some particular problem. ‘‘Normatively appropriate’’

strategies for the solution of some problems are extremely time consuming

and expensive. It may be clear what must be done if one wishes a correct

answer to such problems, but sometimes it may be even clearer that the

correct solution is not worth the effort. This gives rise to more important

questions of normativeness which are not fundamentally empirical in

nature: How much effort, for what kinds of problems, should be expended

to obtain a correct solution?

We have become excited by such normative questions and are pleased that

our book highlights them. We have not been able to make much progress

toward their solution, however. . . . It is our hope that others, particularly

philosophers who are more comfortable with such questions, will be motivated

to pursue them. (Nisbett and Ross 1980, 13 –14)

It is rare for scientists to call on philosophers to contribute in substantive

ways to their scientific projects. Rarer still for scientists who are at the top

of their field.

Armed with the suspicion that there was something useful for philosophers

to do in this area, we organized a symposium at the 2000 Philosophy

of Science Association meeting in Vancouver. The purpose of the

symposium was to explore the connections between research on predictive

modeling and philosophy (see Dawes 2002, Faust and Meehl 2002, Bishop

and Trout 2002). We have also presented these ideas to a number of

audiences at Bryn Mawr College, California State University at Long Beach,

Howard University, Northwestern University, University of Illinois, University

of Innsbruck, University of Utah, and Washington University in St.

Louis. In almost every venue, there were philosophers whose reaction to

these issues was similar to our own: the normative issues raised by the

psychological literature are interesting and important, but analytic epistemology

does not have the resources to adequately address them. So we sat

down to write this book.

Our goal in this book is to bring whatever philosophical expertise we

can to bear on the sorts of normative issues that bedevil psychologists (like

Nisbett and Ross). With a few notable exceptions, the normative concerns

of epistemologists and psychologists have inhabited different intellectual

worlds. When philosophers do discuss psychological findings, it is usually

to dismiss them as irrelevant to epistemology. This book will have achieved

its goals if it leads at least some philosophers and psychologists to admit

(even if ever so grudgingly) that their field of study would benefit from

closer cooperation with their sister discipline.

This book is the product of somewhat unusual philosophical training.

But then again, our philosophy teachers were an unusual collection of

curiosity, talent, and trust. We are grateful to Richard Boyd, Philip

Kitcher, Robert Stalnaker, and Stephen Stich, each of whom taught us in

his own way the value of pursuing interesting but risky projects. They also

encouraged us to explore issues that lie outside the disciplinary confines

of philosophy. In so doing, we were lucky to learn psychology, in graduate

school and after, from Frank Keil, Richard Nisbett, David Pisoni,

V. S. Ramachandran, Robert Remez, Roger Shepard, and Gary Wells.

These psychologists instilled in us an appreciation for a science of the

mind and (probably unwittingly) a recognition of its relevance to philosophical

questions.

Given our general outlook, we doubt that we can very reliably identify

the most important intellectual influences on our epistemological views.

But we are confident that they include Richard Boyd’s ‘‘Scientific Realism

and Naturalistic Epistemology’’ (1980), Alvin Goldman’s ‘‘Epistemics: The

Regulative Theory of Cognition’’ (1978) as well as Epistemology and Cognition

(1986), Philip Kitcher’s ‘‘The Naturalist’s Return’’ (1992) as well as

chapter 8 of The Advancement of Science (1993), Hilary Kornblith’s Inductive

Inference and Its Natural Ground (1993), and Stephen Stich’s The

Fragmentation of Reason (1990).

For useful conversations about the material in this book, we would

like to thank our colleagues and friends: Paul Abela, Robert Baum, Travis

Butler, Douglas Epperson, Joe Kupfer, Dominic Murphy, Gary Pavela, Bill

Robinson, Abe Schwab, Peter Vranas, Daniel Weiskopf, and Gary Wells.

We would like especially to thank Joe Mendola, Michael Strevens, and

Mark Wunderlich, who gave us detailed comments on earlier drafts of this

book, and James Twine, who supplied excellent research assistance. We are

also grateful to the National Science Foundation for grants SES#0354536

(to MB) and SES#0327104 (to JDT) in support of the research culminating

in this book. The findings in Arkes (2003) should keep us modest.

Epistemology and the Psychology

of Human Judgment

This page intentionally left blank

Introduction

The first three chapters of the book introduce the basic building blocks

of our epistemological approach and of our epistemological theory.

Chapter 1 introduces the basic motives and methods of our epistemology.

The goal is to give the reader a clear conception of our overall project. As a

result, the opening chapter is not weighed down with arguments and

qualifications—that comes later. Chapter 2 introduces Statistical Prediction

Rules (SPRs) and offers an explanation for their success. SPRs are

simple, formal rules that have been shown to be at least as reliable, and

typically more reliable, than the predictions of human experts on a wide variety

of problems. On the basis of testable results, psychology can make

normative recommendations about how we ought to reason. We dub the

branches of psychology that provide normative recommendations ‘‘Ameliorative

Psychology.’’ Ameliorative Psychology recommends SPRs on the

basis of testable results: SPRs are reliable, they tend to be easy to use, and

they typically address significant issues. In addition, taking seriously the

success of SPRs requires us to impose discipline on a human mind that is

much too easily tempted by appealing distractions. Certain lines of evidence,

no matter how subjectively attractive or how consecrated by the

concepts central to our epistemological tradition, are to be ignored except

in extreme cases. In chapter 3, we identify some of the basic building

blocks of the epistemological framework that supports the recommendations

of Ameliorative Psychology. The framework assesses the epistemic

merit of reasoning strategies in terms of their robust reliability, their

feasibility and the significance of the problems they tackle. We then argue

that this framework offers a new way to think about applied epistemology.

In particular, it suggests that there are four and only four ways for people

to improve their reasoning.

The middle three chapters of the book (4, 5, and 6) articulate the

central features of our theory of epistemic excellence, Strategic Reliabilism.

Strategic Reliabilism holds that epistemic excellence involves the efficient

allocation of cognitive resources to robustly reliable reasoning strategies

applied to significant problems. Chapter 4 takes up what is the central

notion of Strategic Reliabilism: what it is for a reasoning strategy to be

robustly reliable. Chapter 5 defends a cost-benefit approach to epistemology

and offers an account of what it is for cognitive resources to be

allocated efficiently. Chapter 6 argues that a genuinely normative epistemological

theory must include some notion of significance, and it addresses

the issue of what it is for a problem to be significant.

The final three substantive chapters of the book (7, 8, and 9) put our

views about epistemology to work. In chapter 7, we criticize the approach

that has dominated English-speaking philosophy over the past half-century

or so—what we call Standard Analytic Epistemology (SAE). SAE names a

contingently clustered set of methods and motives. By comparing our

approach to that of SAE, chapter 7 identifies some of the troubles with

SAE and argues that they are serious enough to motivate a radically different

approach to epistemology. Chapter 8 takes Strategic Reliabilism,

which has been extracted from psychology, and turns it back on psychology.

We use Strategic Reliabilism to resolve two debates about whether

certain experimental findings demonstrate deep and systematic failures of

human reasoning. This chapter illustrates one of the main benefits of our

approach to epistemology: it can be used to adjudicate disputes that arise

in psychology that are, at bottom, normative epistemological disputes

about the nature of good reasoning. Chapter 9 attempts to consolidate

some of the lessons of Ameliorative Psychology with some handy heuristics

and illustrative injunctions. We explore the empirical research that

shows how we can enhance the accuracy of diagnostic reasoning, reduce

overconfidence, avoid the regression fallacy, improve our policy assessments,

and restrain the unbridled story-telling surrounding rare or unusual

events. We have no doubt that many significant problems we face

are best addressed with institutional measures, and on this issue much

research remains to be done. But for those problems tractable to voluntary

reasoning strategies, the simple strategies recommended in this chapter

can improve reasoning at low cost and high fidelity.

Chapter 10 briefly sums up our view and points to some of the

challenges that remain in the construction of a naturalistic epistemology.

The Appendix considers 11 objections that we expect philosophers to

level against our views. Some will undoubtedly complain that we have

missed some serious objections, or that our replies to the objections we

do consider are by no means conclusive. Granted. But our goal in the

Appendix is not the wildly ambitious one of overcoming all serious objections.

Instead, our aim is to offer some sense of the resources available

to the naturalist for overcoming what many proponents of SAE are likely

to consider devastating objections.

This book began in our classrooms. At some point, we discovered that

we both teach critical thinking courses that are idiosyncratic in the

same ways—in short, as though they are courses in the psychology of

judgment. For example, we both had our students read Robyn Dawes’s

House of Cards (1994) and Thomas Gilovich’s How We Know What Isn’t So

(1991). We independently arrived at the idea that there were epistemological

lessons to be drawn not just from the heuristics and biases tradition

(which has received attention from philosophers) but also from the fascinating

research on linear predictive modeling. But we also recognized

that psychologists for too long had been wrestling with normative, epistemic

issues with much too little useful input from philosophers. In their

classic book Human Inference: Strategies and Shortcomings of Social Judgment

(1980), Richard Nisbett and Lee Ross call for greater participation

from philosophers in tackling the normative issues that arise in psychology.

[W]e have become increasingly aware of the difficulty of defining what is

‘‘normative’’ when one moves beyond the relatively simple question of how

to solve correctly some particular problem. ‘‘Normatively appropriate’’

strategies for the solution of some problems are extremely time consuming

and expensive. It may be clear what must be done if one wishes a correct

answer to such problems, but sometimes it may be even clearer that the

correct solution is not worth the effort. This gives rise to more important

questions of normativeness which are not fundamentally empirical in

nature: How much effort, for what kinds of problems, should be expended

to obtain a correct solution?

We have become excited by such normative questions and are pleased that

our book highlights them. We have not been able to make much progress

toward their solution, however. . . . It is our hope that others, particularly

philosophers who are more comfortable with such questions, will be motivated

to pursue them. (Nisbett and Ross 1980, 13 –14)

It is rare for scientists to call on philosophers to contribute in substantive

ways to their scientific projects. Rarer still for scientists who are at the top

of their field.

Armed with the suspicion that there was something useful for philosophers

to do in this area, we organized a symposium at the 2000 Philosophy

of Science Association meeting in Vancouver. The purpose of the

symposium was to explore the connections between research on predictive

modeling and philosophy (see Dawes 2002, Faust and Meehl 2002, Bishop

and Trout 2002). We have also presented these ideas to a number of

audiences at Bryn Mawr College, California State University at Long Beach,

Howard University, Northwestern University, University of Illinois, University

of Innsbruck, University of Utah, and Washington University in St.

Louis. In almost every venue, there were philosophers whose reaction to

these issues was similar to our own: the normative issues raised by the

psychological literature are interesting and important, but analytic epistemology

does not have the resources to adequately address them. So we sat

down to write this book.

Our goal in this book is to bring whatever philosophical expertise we

can to bear on the sorts of normative issues that bedevil psychologists (like

Nisbett and Ross). With a few notable exceptions, the normative concerns

of epistemologists and psychologists have inhabited different intellectual

worlds. When philosophers do discuss psychological findings, it is usually

to dismiss them as irrelevant to epistemology. This book will have achieved

its goals if it leads at least some philosophers and psychologists to admit

(even if ever so grudgingly) that their field of study would benefit from

closer cooperation with their sister discipline.

This book is the product of somewhat unusual philosophical training.

But then again, our philosophy teachers were an unusual collection of

curiosity, talent, and trust. We are grateful to Richard Boyd, Philip

Kitcher, Robert Stalnaker, and Stephen Stich, each of whom taught us in

his own way the value of pursuing interesting but risky projects. They also

encouraged us to explore issues that lie outside the disciplinary confines

of philosophy. In so doing, we were lucky to learn psychology, in graduate

school and after, from Frank Keil, Richard Nisbett, David Pisoni,

V. S. Ramachandran, Robert Remez, Roger Shepard, and Gary Wells.

These psychologists instilled in us an appreciation for a science of the

mind and (probably unwittingly) a recognition of its relevance to philosophical

questions.

Given our general outlook, we doubt that we can very reliably identify

the most important intellectual influences on our epistemological views.

But we are confident that they include Richard Boyd’s ‘‘Scientific Realism

and Naturalistic Epistemology’’ (1980), Alvin Goldman’s ‘‘Epistemics: The

Regulative Theory of Cognition’’ (1978) as well as Epistemology and Cognition

(1986), Philip Kitcher’s ‘‘The Naturalist’s Return’’ (1992) as well as

chapter 8 of The Advancement of Science (1993), Hilary Kornblith’s Inductive

Inference and Its Natural Ground (1993), and Stephen Stich’s The

Fragmentation of Reason (1990).

For useful conversations about the material in this book, we would

like to thank our colleagues and friends: Paul Abela, Robert Baum, Travis

Butler, Douglas Epperson, Joe Kupfer, Dominic Murphy, Gary Pavela, Bill

Robinson, Abe Schwab, Peter Vranas, Daniel Weiskopf, and Gary Wells.

We would like especially to thank Joe Mendola, Michael Strevens, and

Mark Wunderlich, who gave us detailed comments on earlier drafts of this

book, and James Twine, who supplied excellent research assistance. We are

also grateful to the National Science Foundation for grants SES#0354536

(to MB) and SES#0327104 (to JDT) in support of the research culminating

in this book. The findings in Arkes (2003) should keep us modest.

Epistemology and the Psychology

of Human Judgment

This page intentionally left blank

Introduction

The first three chapters of the book introduce the basic building blocks

of our epistemological approach and of our epistemological theory.

Chapter 1 introduces the basic motives and methods of our epistemology.

The goal is to give the reader a clear conception of our overall project. As a

result, the opening chapter is not weighed down with arguments and

qualifications—that comes later. Chapter 2 introduces Statistical Prediction

Rules (SPRs) and offers an explanation for their success. SPRs are

simple, formal rules that have been shown to be at least as reliable, and

typically more reliable, than the predictions of human experts on a wide variety

of problems. On the basis of testable results, psychology can make

normative recommendations about how we ought to reason. We dub the

branches of psychology that provide normative recommendations ‘‘Ameliorative

Psychology.’’ Ameliorative Psychology recommends SPRs on the

basis of testable results: SPRs are reliable, they tend to be easy to use, and

they typically address significant issues. In addition, taking seriously the

success of SPRs requires us to impose discipline on a human mind that is

much too easily tempted by appealing distractions. Certain lines of evidence,

no matter how subjectively attractive or how consecrated by the

concepts central to our epistemological tradition, are to be ignored except

in extreme cases. In chapter 3, we identify some of the basic building

blocks of the epistemological framework that supports the recommendations

of Ameliorative Psychology. The framework assesses the epistemic

merit of reasoning strategies in terms of their robust reliability, their

feasibility and the significance of the problems they tackle. We then argue

that this framework offers a new way to think about applied epistemology.

In particular, it suggests that there are four and only four ways for people

to improve their reasoning.

The middle three chapters of the book (4, 5, and 6) articulate the

central features of our theory of epistemic excellence, Strategic Reliabilism.

Strategic Reliabilism holds that epistemic excellence involves the efficient

allocation of cognitive resources to robustly reliable reasoning strategies

applied to significant problems. Chapter 4 takes up what is the central

notion of Strategic Reliabilism: what it is for a reasoning strategy to be

robustly reliable. Chapter 5 defends a cost-benefit approach to epistemology

and offers an account of what it is for cognitive resources to be

allocated efficiently. Chapter 6 argues that a genuinely normative epistemological

theory must include some notion of significance, and it addresses

the issue of what it is for a problem to be significant.

The final three substantive chapters of the book (7, 8, and 9) put our

views about epistemology to work. In chapter 7, we criticize the approach

that has dominated English-speaking philosophy over the past half-century

or so—what we call Standard Analytic Epistemology (SAE). SAE names a

contingently clustered set of methods and motives. By comparing our

approach to that of SAE, chapter 7 identifies some of the troubles with

SAE and argues that they are serious enough to motivate a radically different

approach to epistemology. Chapter 8 takes Strategic Reliabilism,

which has been extracted from psychology, and turns it back on psychology.

We use Strategic Reliabilism to resolve two debates about whether

certain experimental findings demonstrate deep and systematic failures of

human reasoning. This chapter illustrates one of the main benefits of our

approach to epistemology: it can be used to adjudicate disputes that arise

in psychology that are, at bottom, normative epistemological disputes

about the nature of good reasoning. Chapter 9 attempts to consolidate

some of the lessons of Ameliorative Psychology with some handy heuristics

and illustrative injunctions. We explore the empirical research that

shows how we can enhance the accuracy of diagnostic reasoning, reduce

overconfidence, avoid the regression fallacy, improve our policy assessments,

and restrain the unbridled story-telling surrounding rare or unusual

events. We have no doubt that many significant problems we face

are best addressed with institutional measures, and on this issue much

research remains to be done. But for those problems tractable to voluntary

reasoning strategies, the simple strategies recommended in this chapter

can improve reasoning at low cost and high fidelity.

Chapter 10 briefly sums up our view and points to some of the

challenges that remain in the construction of a naturalistic epistemology.

The Appendix considers 11 objections that we expect philosophers to

level against our views. Some will undoubtedly complain that we have

missed some serious objections, or that our replies to the objections we

do consider are by no means conclusive. Granted. But our goal in the

Appendix is not the wildly ambitious one of overcoming all serious objections.

Instead, our aim is to offer some sense of the resources available

to the naturalist for overcoming what many proponents of SAE are likely

to consider devastating objections.