3. The importance of real reliability scores

К оглавлению1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 
85 86 87 88 89 90 91 92 93 94 

As a practical matter, real reliability scores are less useful than they might

seem to be—even for a reliabilist view like ours. The reason is that what

we’re typically interested in when we’re doing applied epistemology is not a

reasoning strategy’s absolute reliability score, but its score relative to other

strategies for solving the same (or some of the same) problems (and

sometimes for solving different problems). And we acquire evidence that

76 Epistemology and the Psychology of Human Judgment

can justify our confidence that one strategy’s reliability score really is higher

than another’s, even without knowing exactly what either strategy’s real score

is. For example, we have a lot of evidence for thinking that Goldberg’s Rule

is more reliable than any alternative rules we know about for distinguishing

between psychotics and neurotics among psychiatric patients on the basis

of MMPI profiles. This evidence is based on the relevant rules having been

applied literally thousands of times to various populations of psychiatric

patients. Further, the Goldberg Rule is easier to use than most of its competitors,

and there is no special reason to suppose the causal structure of

the relevant parts of the world is about to change in a way that would

undercut the rule’s reliability. This overwhelming evidence is all we really

need to make a reasonable epistemic recommendation.

We do not need to know precisely what a strategy’s real reliability score

is in order to do good applied epistemology. But we should not derogate

real reliability scores too much. Real reliability scores are a theoretical and

unobservable posit of our epistemological theory. They play a vital role in

at least two places. First, a reasoning strategy’s observed reliability score is

supposed to approximate its real score. So when figuring out a strategy’s

observed score—its track record—we are inevitably guided by our notion

of its real reliability score. (This is why, for example, we don’t assign observed

reliability scores after testing an empirical reasoning strategy on just

one problem.) A strategy’s real reliability score is like the statistical notion

of a ‘‘real value,’’ such as a population mean. We explain features of the

sample from a population in terms of a ‘‘real value’’ of the population, a

value that the population has independent of attempts to measure it. The

second role real reliability scores play in epistemology is as (part of) the

ultimate ground of our epistemic judgments. The ultimate reason Goldberg’s

Rule is the best strategy we can use on the MMPI prediction problems

is that its real reliability score is higher than alternatives (and it is at

least as easy to use as its alternatives). Our epistemic recommendations are

based (in part) on the quality of the reasons we have for believing claims

about the real reliability scores of various reasoning strategies.

As a practical matter, real reliability scores are less useful than they might

seem to be—even for a reliabilist view like ours. The reason is that what

we’re typically interested in when we’re doing applied epistemology is not a

reasoning strategy’s absolute reliability score, but its score relative to other

strategies for solving the same (or some of the same) problems (and

sometimes for solving different problems). And we acquire evidence that

76 Epistemology and the Psychology of Human Judgment

can justify our confidence that one strategy’s reliability score really is higher

than another’s, even without knowing exactly what either strategy’s real score

is. For example, we have a lot of evidence for thinking that Goldberg’s Rule

is more reliable than any alternative rules we know about for distinguishing

between psychotics and neurotics among psychiatric patients on the basis

of MMPI profiles. This evidence is based on the relevant rules having been

applied literally thousands of times to various populations of psychiatric

patients. Further, the Goldberg Rule is easier to use than most of its competitors,

and there is no special reason to suppose the causal structure of

the relevant parts of the world is about to change in a way that would

undercut the rule’s reliability. This overwhelming evidence is all we really

need to make a reasonable epistemic recommendation.

We do not need to know precisely what a strategy’s real reliability score

is in order to do good applied epistemology. But we should not derogate

real reliability scores too much. Real reliability scores are a theoretical and

unobservable posit of our epistemological theory. They play a vital role in

at least two places. First, a reasoning strategy’s observed reliability score is

supposed to approximate its real score. So when figuring out a strategy’s

observed score—its track record—we are inevitably guided by our notion

of its real reliability score. (This is why, for example, we don’t assign observed

reliability scores after testing an empirical reasoning strategy on just

one problem.) A strategy’s real reliability score is like the statistical notion

of a ‘‘real value,’’ such as a population mean. We explain features of the

sample from a population in terms of a ‘‘real value’’ of the population, a

value that the population has independent of attempts to measure it. The

second role real reliability scores play in epistemology is as (part of) the

ultimate ground of our epistemic judgments. The ultimate reason Goldberg’s

Rule is the best strategy we can use on the MMPI prediction problems

is that its real reliability score is higher than alternatives (and it is at

least as easy to use as its alternatives). Our epistemic recommendations are

based (in part) on the quality of the reasons we have for believing claims

about the real reliability scores of various reasoning strategies.