3. The importance of real reliability scores
К оглавлению1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1617 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94
As a practical matter, real reliability scores are less useful than they might
seem to be—even for a reliabilist view like ours. The reason is that what
we’re typically interested in when we’re doing applied epistemology is not a
reasoning strategy’s absolute reliability score, but its score relative to other
strategies for solving the same (or some of the same) problems (and
sometimes for solving different problems). And we acquire evidence that
76 Epistemology and the Psychology of Human Judgment
can justify our confidence that one strategy’s reliability score really is higher
than another’s, even without knowing exactly what either strategy’s real score
is. For example, we have a lot of evidence for thinking that Goldberg’s Rule
is more reliable than any alternative rules we know about for distinguishing
between psychotics and neurotics among psychiatric patients on the basis
of MMPI profiles. This evidence is based on the relevant rules having been
applied literally thousands of times to various populations of psychiatric
patients. Further, the Goldberg Rule is easier to use than most of its competitors,
and there is no special reason to suppose the causal structure of
the relevant parts of the world is about to change in a way that would
undercut the rule’s reliability. This overwhelming evidence is all we really
need to make a reasonable epistemic recommendation.
We do not need to know precisely what a strategy’s real reliability score
is in order to do good applied epistemology. But we should not derogate
real reliability scores too much. Real reliability scores are a theoretical and
unobservable posit of our epistemological theory. They play a vital role in
at least two places. First, a reasoning strategy’s observed reliability score is
supposed to approximate its real score. So when figuring out a strategy’s
observed score—its track record—we are inevitably guided by our notion
of its real reliability score. (This is why, for example, we don’t assign observed
reliability scores after testing an empirical reasoning strategy on just
one problem.) A strategy’s real reliability score is like the statistical notion
of a ‘‘real value,’’ such as a population mean. We explain features of the
sample from a population in terms of a ‘‘real value’’ of the population, a
value that the population has independent of attempts to measure it. The
second role real reliability scores play in epistemology is as (part of) the
ultimate ground of our epistemic judgments. The ultimate reason Goldberg’s
Rule is the best strategy we can use on the MMPI prediction problems
is that its real reliability score is higher than alternatives (and it is at
least as easy to use as its alternatives). Our epistemic recommendations are
based (in part) on the quality of the reasons we have for believing claims
about the real reliability scores of various reasoning strategies.
As a practical matter, real reliability scores are less useful than they might
seem to be—even for a reliabilist view like ours. The reason is that what
we’re typically interested in when we’re doing applied epistemology is not a
reasoning strategy’s absolute reliability score, but its score relative to other
strategies for solving the same (or some of the same) problems (and
sometimes for solving different problems). And we acquire evidence that
76 Epistemology and the Psychology of Human Judgment
can justify our confidence that one strategy’s reliability score really is higher
than another’s, even without knowing exactly what either strategy’s real score
is. For example, we have a lot of evidence for thinking that Goldberg’s Rule
is more reliable than any alternative rules we know about for distinguishing
between psychotics and neurotics among psychiatric patients on the basis
of MMPI profiles. This evidence is based on the relevant rules having been
applied literally thousands of times to various populations of psychiatric
patients. Further, the Goldberg Rule is easier to use than most of its competitors,
and there is no special reason to suppose the causal structure of
the relevant parts of the world is about to change in a way that would
undercut the rule’s reliability. This overwhelming evidence is all we really
need to make a reasonable epistemic recommendation.
We do not need to know precisely what a strategy’s real reliability score
is in order to do good applied epistemology. But we should not derogate
real reliability scores too much. Real reliability scores are a theoretical and
unobservable posit of our epistemological theory. They play a vital role in
at least two places. First, a reasoning strategy’s observed reliability score is
supposed to approximate its real score. So when figuring out a strategy’s
observed score—its track record—we are inevitably guided by our notion
of its real reliability score. (This is why, for example, we don’t assign observed
reliability scores after testing an empirical reasoning strategy on just
one problem.) A strategy’s real reliability score is like the statistical notion
of a ‘‘real value,’’ such as a population mean. We explain features of the
sample from a population in terms of a ‘‘real value’’ of the population, a
value that the population has independent of attempts to measure it. The
second role real reliability scores play in epistemology is as (part of) the
ultimate ground of our epistemic judgments. The ultimate reason Goldberg’s
Rule is the best strategy we can use on the MMPI prediction problems
is that its real reliability score is higher than alternatives (and it is at
least as easy to use as its alternatives). Our epistemic recommendations are
based (in part) on the quality of the reasons we have for believing claims
about the real reliability scores of various reasoning strategies.