How Not to Detect Design: A review of William Dembski’s “The Design Inference”
Branden Fitelson, Christopher Stephens, and Elliott Sober review
William Dembski’s 1998 book __Design Inference: Eliminating Chance Through Small Probabilities__(Cambridge: Cambridge University Press. 1998. xvii + 243 pg. ISBN 0-521-62387-1).
How Not to Detect Design*
As every philosopher knows, “the design argument” concludes that God exists from premises that cite the adaptive complexity of organisms or the lawfulness and orderliness of the whole universe. Since 1859, it has formed the intellectual heart of creationist opposition to the Darwinian hypothesis that organisms evolved their adaptive features by the mindless process of natural selection. Although the design argument developed as a defense of theism, the logic of the argument in fact encompasses a larger set of issues. William Paley saw clearly that we sometimes have an excellent reason to postulate the existence of an intelligent designer. If we find a watch on the heath, we reasonably infer that it was produced by an intelligent watchmaker. This design argument makes perfect sense. Why is it any different to claim that the eye was produced by an intelligent designer? Both critics and defenders of the design argument need to understand what the ground rules are for inferring that an intelligent designer is the unseen cause of an observed effect.
Dembski’s book is an attempt to clarify these ground rules. He proposes a procedure for detecting design and discusses how it applies to a number of mundane and nontheological examples, which more or less resemble Paley’s watch. Although the book takes no stand on whether creationism is more or less plausible than evolutionary theory, Dembski’s epistemology can be evaluated without knowing how he thinks it bears on this highly charged topic. In what follows, we will show that Dembski’s account of design inference is deeply flawed. Sometimes he is too hard on hypotheses of intelligent design; at other times he is too lenient. Neither creationists nor evolutionists nor people who are trying to detect design in nontheological contexts should adopt Dembski’s framework.
The Explanatory Filter
Dembski’s book provides a series of representations of how design inference works. The exposition starts simple and grows increasingly complex. However, the basic pattern of analysis can be summarized as follows. Dembski proposes an “explanatory filter” (37), which is a procedure for deciding how best to explain an observation E:
(2) The Regularity hypothesis is more parsimonious than Chance, and Chance is more parsimonious than Design. To evaluate these alternatives, begin with the most parsimonious possibility and move down the list until you reach an explanation you can accept.
(3) If E has a high probability, you should accept Regularity; otherwise, reject Regularity and move down the list.
(4) If the Chance hypothesis assigns E a sufficiently low probability and E is “specified,” then reject Chance and move down the list; otherwise, accept Chance.
(5) If you have rejected Regularity and Chance, then you should accept Design as the explanation of E.
The entire book is an elaboration of the ideas that comprise the Explanatory Filter. Notice that the filter is eliminativist, with the Design hypothesis occupying a special position.
We have interpreted the Filter as sometimes recommending that you should accept Regularity or Chance. This is supported, for example, by Dembski’s remark (38) that “if E happens to be an HP [a high probability] event, we stop and attribute E to a regularity.” However, some of the circumlocutions that Dembski uses suggest that he doesn’t think you should ever “accept” Regularity or Chance. The most you should do is “not reject” them. Under this alternative interpretation, Dembski is saying that if you fail to reject Regularity, you can believe any of the three hypotheses, or remain agnostic about all three. And if you reject Regularity, but fail to reject Chance, you can believe either Chance or Design, or remain agnostic about them both. Only if you have rejected Regularity and Chance must you accept one of the three, namely Design. Construed in this way, a person who believes that every event is the result of Design has nothing to fear from the Explanatory Filter — no evidence can ever dislodge that opinion. This may be Dembski’s view, but for the sake of charity, we have described the Filter in terms of rejection and acceptance.
The Caputo Example
Before discussing the filter in detail, we want to describe Dembski’s treatment of one of the main examples that he uses to motivate his analysis (9-19,162-166). This is the case of Nicholas Caputo, who was a member of the Democratic party in New Jersey. Caputo’s job was to determine whether Democrats or Republicans would be listed first on the ballot. The party listed first in an election has an edge, and this was common knowledge in Caputo’s day. Caputo had this job for 41 years and he was supposed to do it fairly. Yet, in 40 out of 41 elections, he listed the Democrats first. Caputo claimed that each year he determined the order by drawing from an urn that gave Democrats and Republicans the same chance of winning. In spite of his protestations, Caputo was brought up on charges and the judges found against him. They rejected his claim that the outcome was due to chance, and were persuaded that he had rigged the results. The ordering of names on the ballots was due to Caputo’s intelligent design.
In this story, the hypotheses of Chance and Intelligent Design are prominent. But what of the first alternative, that of Regularity? Dembski (11) says that this can be rejected because our background knowledge tells us that Caputo probably didn’t innocently use a biased process. For example, we can rule out the possibility that Caputo, with the most honest of intentions, spun a roulette wheel in which 00 was labeled “Republican” and all the other numbers were labeled “Democrat.” Apparently, we know before we examine Caputo’s 41 decisions that there are just two possibilities — he did the equivalent of tossing a fair coin (Chance) or he intentionally gave the edge to his own party (Design).
There is a straightforward reason for thinking that the observed outcomes favor Design over Chance. If Caputo had allowed his political allegiance to guide his arrangement of ballots, you’d expect Democrats to be listed first on all or almost all of the ballots. However, if Caputo did the equivalent of tossing a fair coin, the outcome he obtained would be very surprising. This simple analysis also can be used to represent Paley’s argument about the watch (Sober 1993). The key concept is likelihood. The likelihood of a hypothesis is the probability it confers on the observations; it is not the probability that the observations confer on the hypothesis. The likelihood of H relative to E is Pr(E*H), not Pr(H*E). Chance and Design can be evaluated by comparing their likelihoods, relative to the same set of observations. We do not claim that likelihood is the whole story, but surely it is relevant.
The reader will notice that the Filter does not use this simple likelihood analysis to help decide between Chance and Design. The likelihood of Chance is considered, but the likelihood of Design never is. Instead, the Chance hypothesis is evaluated for properties additional to its likelihood. Dembski thinks it is possible to reject Chance and accept Design without asking what Design predicts. Whether the Filter succeeds in showing that this possible is something we’ll have to determine.
The Three Alternative Explanations
Dembski defines the Regularity hypothesis in different ways. Sometimes it is said to assert that the evidence E is noncontingent and is reducible to law (39, 53); at other times it is taken to claim that E is a deterministic consequence of earlier conditions (65, 146n5); and at still other times, it is supposed to say that E was highly probable, given some earlier state of the world (38). The Chance Hypothesis is taken to assign to E a lower probability than the Regularity Hypothesis assigns (40). The Design Hypothesis is said to be the complement of the first two alternatives. As a matter of stipulation, the three hypotheses are mutually exclusive and collectively exhaustive (36).
Dembski emphasizes that design need not involve intelligent agency (8-9, 36, 60, 228- 229). He regards design as a mark of intelligent agency; intelligent agency can produce design, but he seems to think that there could be other causes as well. On the other hand, Dembski says that “the explanatory filter pinpoints how we recognize intelligent agency (66)” and his section 2.4 is devoted to showing that design is reliably correlated with intelligent agency. Dembski needs to supply an account of what he means by design and how it can be caused by something other than intelligent agency. His vague remark (228-229) that design is equivalent to “information” is not enough. Dembski quotes Dretske (1981) with approval, as deploying the concept of information that the design hypothesis uses. However, Dretske’s notion of information is, as Dembski points out, the Shannon-Weaver account, which describes a probabilistic dependency between two events labeled source and receiver. Hypotheses of mindless chance can be stated in terms of the Shannon-Weaver concept. Dembski (39) also says that the design hypothesis isn’t “characterized by probability.”
Understanding what “regularity,” “chance,” and “design” mean in Dembski’s framework is made more difficult by some of his examples. Dembski discusses a teacher who finds that the essays submitted by two students are nearly identical (46). One hypothesis is that the students produced their work independently; a second hypothesis asserts that there was plagiarism. Dembski treats the hypothesis of independent origination as a Chance hypothesis and the plagiarism hypothesis as an instance of Design. Yet, both describe the matching papers as issuing from intelligent agency, as Dembski points out (47). Dembski says that context influences how a hypothesis gets classified (46). How context induces the classification that Dembski suggests remains a mystery.
The same sort of interpretive problem attaches to Dembski’s discussion of the Caputo example. We think that all of the following hypotheses appeal to intelligent agency: (i) Caputo decided to spin a roulette wheel on which 00 was labeled “Republican” and the other numbers were labeled “Democrat;” (ii) Caputo decided to toss a fair coin; (iii) Caputo decided to favor his own party. Since all three hypotheses describe the ballot ordering as issuing from intelligent agency, all, apparently, are instances of Design in Dembski’s sense. However, Dembski says that they are examples, respectively, of Regularity, Chance, and Design.
The Parsimony Ordering
Dembski says that Regularity is a more parsimonious hypothesis than Chance, and that Chance is more parsimonious than Design (38-39). He defends this ordering as follows:
Here Dembski seems to interpret Regularity to mean that E [an observed event] is nomologically necessary or that E is a deterministic consequence of initial conditions. Still, why does this show that Regularity is simpler than Chance? And why is Chance simpler than Design? Even if design hypotheses were “not characterized by probability,” why would that count as a reason? But, in fact, design hypotheses do in many instances confer probabilities on the observations. The ordering of Democrats and Republicans on the ballots is highly probable, given the hypothesis that Caputo rigged the ballots to favor his own party. Dembski supplements this general argument for his parsimony ordering with two examples (39). Even if these examples were convincing, they would not establish the general point about the parsimony ordering.
It may be possible to replace Dembski’s faulty argument for his parsimony ordering with a different argument that comes close to delivering what he wants. Perhaps determinism can be shown to be more parsimonious than indeterminism (Sober 1999a) and perhaps explanations that appeal to mindless processes can be shown to be simpler than explanations that appeal to intelligent agency (Sober 1998). But even if this can be done, it is important to understand what this parsimony ordering means. When scientists choose between competing curves, the simplicity of the competitors matters, but so does their fit-to-data. You don’t reject a simple curve and adopt a complex curve just by seeing how the simple curve fits the data and without asking how well the complex curve does so. You need to ask how well both hypotheses fit the data. Fit-to-data is important in curve-fitting because it is a measure of likelihood; curves that are closer to the data confer on the data a higher probability than curves that are more distant. Dembski’s parsimony ordering, even if correct, makes it puzzling why the Filter treats the likelihood of the Chance hypothesis as relevant, but ignores the likelihoods of Regularity and Design.
Why Regularity is Rejected
As just noted, the Explanatory Filter evaluates Regularity and Chance in different ways. The Chance hypothesis is evaluated in part by asking how probable it says the observations are. However, Regularity is not evaluated by asking how probable it says the observations are. The filter starts with the question, “Is E a high probability event (38)?” This doesn’t mean “is E a high probability event according to the Regularity hypothesis?” Rather, you evaluate the probability of E on its own. Presumably, if you observe that events like E occur frequently, you should say that E has a high probability and so should conclude that E is due to Regularity. If events like E rarely occur, you should reject Regularity and move down the list. However, since a given event can be described in many ways, any event can be made to appear common, and any can be made to appear rare.
Dembski’s procedure for evaluating Regularity hypotheses would make no sense if it were intended to apply to specific hypotheses of that kind. After all, specific Regularity hypotheses (e.g., Newtonian mechanics) are often confirmed by events that happen rarely — the return of a comet, for example. And specific Regularity hypotheses are often disconfirmed by events that happen frequently. This suggests that what gets evaluated under the heading of “Regularity” are not specific hypotheses of that kind, but the general claim that E is due to some regularity or other. Understood in this way, it makes more sense why the likelihood of the Regularity hypothesis plays no ro e in the Explanatory Filter. The claim that E is due to some regularity or other, by definition, says that E was highly probable, given antecedent conditions.
It is important to recognize that the Explanatory Filter is enormously ambitious. You don’t just reject a given Regularity hypothesis; you reject all possible Regularity explanations (53). And the same goes for Chance — you reject the whole category; the Filter “sweeps the field clear” of all specific Chance hypotheses (41, 52-53). We doubt that there is any general inferential procedure that can do what Dembski thinks the Filter accomplishes. Of course, you presumably can accept “E is due to some regularity or other” if you accept a specific regularity hypothesis. But suppose you have tested and rejected the various specific regularity hypotheses that your background beliefs suggest. Are you obliged to reject the claim that there exists a regularity hypothesis that explains E? Surely it is clear that this does not follow.
The fact that the Filter allows you to accept or reject Regularity without attending to what specific Regularity hypotheses predict has some peculiar consequences. Suppose you have in mind just one specific regularity hypothesis that is a candidate for explaining E; you think that if E has a regularity-style explanation, this has got to be it. If E is a rare type of event, the Filter says to conclude that E is not due to Regularity. This can happen even if the specific hypothesis, when conjoined with initial condition statements, predicts E with perfect precision. Symmetrically, if E is a common kind of event, the Filter says not to reject Regularity, even if your lone specific Regularity hypothesis deductively entails that E is false. The Filter is too hard on Regularity, and too lenient.
The Specification Condition
To reject Chance, the evidence E must be “specified.” This involves four conditions — CINDE, TRACT, DELIM, and the description D* that you use to delimit E must have a low probability on the Chance hypothesis. We consider these in turn.
Dembski says several times that you can’t reject a Chance hypothesis just because it says that what you observe was improbable. If Jones wins a lottery, you can’t automatically conclude that there is something wrong with the hypothesis that the lottery was fair and that Jones bought just one of the 10,000 tickets sold. To reject Chance, further conditions must be satisfied. CINDE is one of them.
CINDE means conditional independence. This is the requirement that Pr(E* H & I) = Pr(E * H), where H is the Chance hypothesis, E is the observations, and I is your background knowledge. H must render E conditionally independent of I. CINDE requires that H capture everything that your background beliefs say is probabilistically relevant to the occurrence of E.
CINDE is too lenient on Chance hypotheses — it says that their violating CINDE suffices for them to be accepted (or not rejected). Suppose you want to explain why Smith has lung cancer (E). It is part of your background knowledge (I) that he smoked cigarettes for thirty years, but you are considering the hypothesis (H) that Smith read the works of Ayn Rand and that this helped bring about his illness. To investigate this question, you do a statistical study and discover that smokers who read Rand have the same chance of lung cancer as smokers who do not. This study allows you to draw a conclusion about Smith — that Pr(E * H&I) = Pr(E * not-H&I). Surely this equality is evidence against the claim that E is due to H. However, the filter says that you can’t reject the causal claim, because CINDE is false — Pr(E * H&I) ( Pr(E * H).
TRACT and DELIM
The ideas examined so far in the Filter are probabilistic. The TRACT condition introduces concepts from a different branch of mathematics – the theory of computational complexity. TRACT means tractability – to reject the Chance hypothesis, it must be possible for you to use your background information to formulate a description D* of features of the observations E. To construct this description, you needn’t have any reason to think that it might be true. For example, you could satisfy TRACT by obtaining the description of E by “brute force” – that is, by producing descriptions of all the possible outcomes, one of which happens to cover E (150- 151).
Whether you can produce a description depends on the language and computational framework used. For example, the evidence in the Caputo example can be thought of as a specific sequence of 40 Ds and 1 R. TRACT would be satisfied if you have the ability to generate all of the following descriptions: “0 Rs and 41 Ds,” “1 R and 40 Ds,” “2 Rs and 39 Ds,” … “41 Rs and 0 Ds.” Whether you can produce these descriptions depends on the character of the language you use (does it contain those symbols or others with the same meaning?) and on the computational procedures you use to generate descriptions (does generating those descriptions require a small number of steps, or too many for you to perform in your lifetime?). Because tractability depends on your choice of language and computational procedures, we think that TRACT has no evidential significance at all. Caputo’s 41 decisions count against the hypothesis that he used a fair coin, and in favor of the hypothesis that he cheated, for reasons that have nothing to do with TRACT. The relevant point is simply that Pr(E*Chance) << Pr(E*Design). This fact is not relative to the choice of language or computational framework.
The DELIM condition, as far as we can see, adds nothing to TRACT. A description D*, generated by one’s background information, “delimits” the evidence E just in case E entails D*. In the Caputo case, TRACT and DELIM would be satisfied if you were able to write down all possible sequences of D’s and R’s that are 41 letters long. They also would be satisfied by generating a series of weaker descriptions, like the one just mentioned. In fact, just writing down a tautology satisfies TRACT and DELIM (165). On the assumption that human beings are able to write down tautologies, we conclude that these two conditions are always satisfied and so play no substantive role in the Filter.
Do CINDE, TRACT, and DELIM “Call the Chance Hypothesis into Question”?
Dembski argues that CINDE, TRACT and DELIM, if true, “call the chance hypothesis H into question.” We quote his argument in its entirety:
The interrelation between CINDE and TRACT is important. Because I is conditionally independent of E given H, any knowledge S has about I ought to give S no knowledge about E so long as — and this is the crucial assumption — E occurred according to the chance hypothesis H. Hence, any pattern formulated on the basis of I ought not give S any knowledge about E either. Yet the fact that it does in case D delimits E means that I is after all giving S knowledge about E. The assumption that E occurred according to the chance hypothesis H, though not quite refuted, is therefore called into question (147).
Dembski then adds:
We’ll address this claim about the impact of low probability later.
To reconstruct Dembski’s argument, we need to clarify how he understands the conjunction TRACT & DELIM. Dembski says that when TRACT and DELIM are satisfied, your background beliefs I provide you with “knowledge” or “information” about E (143, 147). In fact, TRACT and DELIM have nothing to do with informational relevance understood as an evidential concept. When I provides information about E, it is natural to think that Pr(E | I) – Pr(E); I provides information because taking it into account changes the probability you assign to E. It is easy to see how TRACT & DELIM can both be satisfied by brute force without this evidential condition’s being satisfied. Suppose you have no idea how Caputo might have obtained his sequence of D’s and R’s; still, you are able to generate the sequence of descriptions we mentioned before. The fact that you can generate a description which delimits (or even matches) E does not ensure that your background knowledge provides evidence as to whether E will occur. As noted, generating a tautology satisfies both TRACT and DELIM, but tautologies don’t provide information about E.
Even though the conjunction TRACT & DELIM should not be understood evidentially (i.e., as asserting that Pr[E | I] – Pr[E]), we think this is how Dembski understands TRACT & DELIM in the argument quoted. This suggests the following reconstruction of Dembski’s argument:
(2) If CINDE is true and S is warranted in accepting H (i.e., that E is due to chance), then S should assign Pr(E | I) = Pr(E).
(3) If TRACT and DELIM are true, then S should not assign Pr(E | I) = Pr(E). ———————– (4) Therefore, S is not warranted in accepting H.
Thus reconstructed, Dembski’s argument is valid. We grant premiss (1) for the sake of argument. We’ve already explained why (3) is false. So is premiss (2); it seems to rely on something like the following principle:
(*) If S should assign Pr(E|H&I) = p and S is warranted in accepting H, then S should assign Pr(E|I) = p. If (*) were true, (2) would be true. However, (*) is false. For (*) entails If S should assign Pr(H|H) = 1.0 and S is warranted in accepting H, then S should assign Pr(H) = 1.0.
Justifiably accepting H does not justify assigning H a probability of unity. Bayesians warn against assigning probabilities of 1 and 0 to any proposition that you might want to consider revising later. Dembski emphasizes that the Chance hypothesis is always subject to revision.
It is worth noting that a weaker version of (2) is true:
(2*) If CINDE is true and S should assign Pr(H)=1, then S should assign Pr(E | I) = Pr(E). One then can reasonably conclude that (4*) S should not assign Pr(H) = 1.
However, a fancy argument isn’t needed to show that (4*) is true. Moreover, the fact that (4*) is true does nothing to undermine S’s confidence that the Chance hypothesis H is the true explanation of E, provided that S has not stumbled into the brash conclusion that H is entirely certain. We conclude that Dembski’s argument fails to “call H into question.”
It may be objected that our criticism of Dembski’s argument depends on our taking the conjunction TRACT & DELIM to have probabilistic consequences. We reply that this is a charitable reading of his argument. If the conjunction does not have probabilistic consequences, then the argument is a nonstarter. How can purely non-probabilistic conditions come into conflict with a purely probabilistic condition like CINDE? Moreover, since TRACT and DELIM, sensu strictu, are always true (if the agent’s side information allows him/her to generate a tautology), how could these trivially satisfied conditions, when coupled with CINDE, possibly show that H is questionable?
The Improbability Threshold
The Filter says that Pr(E * Chance) must be sufficiently low if Chance is to be rejected. How low is low enough? Dembski’s answer is that Pr(E(n) * Chance) < 1/2, where n is the number of times in the history of the universe that an event of kind E actually occurs (209, 214-217). As mentioned earlier, if Jones wins a lottery, it does not follow that we should reject the hypothesis that the lottery was fair and that he bought just one of the 10,000 tickets sold. Dembski thinks the reason this is so is that lots of other lotteries have occurred. If p is the probability of Jones’ winning the lottery if it is fair and he bought one of the 10,0000 tickets sold, and if there are n such lotteries that ever occur, then the relevant probability to consider is Pr(E(n) * Chance) = 1 – (1-p)n . If n is large enough this quantity can be greater than 1/2, even though p is very small. As long as the probability exceeds 1/2 that Smith wins lottery L2, or Quackdoodle wins lottery L3, or … or Snerdley wins lottery Ln, given the hypothesis that each of these lotteries was fair and the individuals named each bought one of the 10,000 tickets sold, we shouldn’t reject the Chance hypothesis about Jones.
Why is 1/2 the relevant threshold? Dembski thinks this follows from the Likelihood Principle (190-198). As noted earlier, that principle states that if two hypotheses confer different probabilities on the same observations, the one that entails the higher probability is the one that is better supported by those observations. Dembski thinks this principle solves the following prediction problem. If the Chance hypothesis predicts that either F or not-F will be true, but says that the latter is more probable, then, if you believe the Chance hypothesis and must predict whether F or not-F will be true, you should predict not-F. We agree that if a gun were put to your head, that you should predict the option that the Chance hypothesis says is more probable if you believe the Chance hypothesis and this exhausts what you know that is relevant. However, this doesn’t follow from the likelihood principle. The likelihood principle tells you how to evaluate different hypotheses by seeing what probabilities they confer on the observations. Dembski’s prediction principle describes how you should choose between two predictions, not on the basis of observations, but on the basis of a theory you already accept; the theory says that one prediction is more probable, not that it is more likely.
Even though Dembski’s prediction principle is right, it does not entail that you should reject Chance if Pr(E(n) * Chance) < 1/2 and the other specification conditions are satisfied. Dembski thinks that you face a “probabilistic inconsistency” (196) if you believe the Chance hypothesis and the Chance hypothesis leads you to predict not-F rather than F, but you then discover that E is true and that E is an instance of F. However, there is no inconsistency here of any kind. Perfectly sensible hypotheses sometimes entail that not-F is more probable than F; they can remain perfectly sensible even if F has the audacity to occur.
An additional reason to think that there is no “probabilistic inconsistency” here is that H and not-H can both confer an (arbitrarily) low probability on E. In such cases, Dembski must say that you are caught in a “probabilistic inconsistency” no matter what you accept. Suppose you know that an urn contains either 10% green balls or 1% green balls; perhaps you saw the urn being filled from one of two buckets (you don’t know which), whose contents you examined. Suppose you draw 10 balls from the urn and find that 7 are green. From a likelihood point of view, the evidence favors the 10% hypothesis. However, Dembski would point out that the 10% hypothesis predicted that most of the balls in your sample would fail to be green. Your observation contradicts this prediction. Are you therefore forced to reject the 10% hypothesis? If so, you are forced to reject the 1% hypothesis on the same grounds. But you know that one or the other hypothesis is true. Dembski’s talk of a “probabilistic inconsistency” s ggests that he thinks