[Philip_E._Tetlock]_Expert_Political_Judgment_How(Bookos.org)
Very interesting book!
—-
Game Theorists. The rivalry between Sherlock Holmes and the evil
genius Professor Moriarty illustrates how indeterminacy can arise as a
natural by-product of rational agents second-guessing each other. When
the two first met, Moriarty was eager, too eager, to display his capacity
for interactive thinking by announcing: “All I have to say has already
crossed your mind.” Holmes replied: “Then possibly my answer has
crossed yours.” As the plot unfolds, Holmes uses his superior “interac-
tive knowledge” to outmaneuver Moriarty by unexpectedly getting off
the train at Canterbury, thwarting Moriarty who had calculated that Paris
was Holmes’s rational destination. Convoluted though it is, Moriarty
failed to recognize that Holmes had already recognized that Moriarty
would deduce what a rational Holmes would do under the circum-
stances, and the odds now favored Holmes getting off the train earlier
than once planned.23
Indeterminacy problems of this sort are the bread and butter of behav-
ioral game theory. In the “guess the number” game, for example, con-
testants pick a number between 0 and 100, with the goal of making their
guess come as close as possible to two-thirds of the average guess of all
the contestants.
24 In a world of only rational players—who base their
guesses on the maximum number of levels of deduction—the equilib-
rium is 0. However, in a contest run at Richard Thaler’s prompting by
the Financial Times,
25 the most popular guesses were 33 (the right guess
if everyone else chooses a number at random, producing an average guess
of 50) and 22 (the right guess if everyone thinks through the preceding
argument and picks 33). Dwindling numbers of respondents carried the
deductive logic to the third stage (picking two-thirds of 22) or higher,
with a tiny hypereducated group recognizing the logically correct answer
to be 0. The average guess was 18.91 and the winning guess, 13, which
suggests that, for this newspaper’s readership, a third order of sophisti-
cation was roughly optimal.
interesting
–
Our reluctance to acknowledge unpredictability keeps us looking for
predictive cues well beyond the point of diminishing returns. 39 I witnessed
a demonstration thirty years ago that pitted the predictive abilities of a
classroom of Yale undergraduates against those of a single Norwegian
rat. The task was predicting on which side of a T-maze food would ap-
pear, with appearances determined—unbeknownst to both the humans
and the rat—by a random binomial process (60 percent left and 40 per-
cent right). The demonstration replicated the classic studies by Edwards
and by Estes: the rat went for the more frequently rewarded side (getting
it right roughly 60 percent of the time), whereas the humans looked hard
for patterns and wound up choosing the left or the right side in roughly
the proportion they were rewarded (getting it right roughly 52 percent of
the time). Human performance suffers because we are, deep down, de-
terministic thinkers with an aversion to probabilistic strategies that ac-
cept the inevitability of error. We insist on looking for order in random
sequences. Confronted by the T-maze, we look for subtle patterns like
“food appears in alternating two left/one right sequences, except after
the third cycle when food pops up on the right.” This determination to
ferret out order from chaos has served our species well. We are all bene-
ficiaries of our great collective successes in the pursuit of deterministic reg-
ularities in messy phenomena: agriculture, antibiotics, and countless other
inventions that make our comfortable lives possible. But there are occa-
sions when the refusal to accept the inevitability of error—to acknowledge
that some phenomena are irreducibly probabilistic—can be harmful.
indeed, but generally it is wise to not accept the unpredictability hypothesis about some fenomena. many things that were thought unpredictable for centures turned out to be predictable after all, or at least to some degree. i have confidence we will see the same for earthquakes, weather systems and the like in the future as well.
predictability (and the related determinism) hypothesis are good working hypotheses, even if they turn out to be wrong some times.
this is what i wrote about years ago on my danish blog here. basically, its a 2×2 table:
| What we think/what is true | Determinism | Indeterminism |
| Determinism | We keep looking for explanations for fenomena and in over time, we find regularities and explanations. | We waste time looking for patterns that arent there. |
| Indeterminism | We dont spend time looking for patterns, but there actually are patterns we that cud use to predict the future, and hence we lose out on possible advances in science. | We dont waste time looking for patterns that arent there. |
The above is assuming that indeterminism implies total unpredictability. This isnt true, but in the simplified case where were dealing with completely random fenomena and completely predictable fenomena, this is a reasonable way of looking at it. IMO, it is much better to waste time looking for explanations for things that are not orderly (after all), than risk not spotting real patterns in nature.
Finally, regardless of whether it is rash to abandon the meliorist search
for the Holy Grail of good judgment, most of us feel it is. When we weigh
the perils of Type I errors (seeking correlates of good judgment that will
prove ephemeral) against those of Type II errors (failing to discover
durable correlates with lasting value), it does not feel like a close call. We
would rather risk anointing lucky fools over ignoring wise counsel. Radi-
cal skepticism is too bitter a doctrinal pill for most of us to swallow.
exactly
–
But betting is one thing, paying up another. Focusing just on reactions
to losing reputational bets, figure 4.1 shows that neither hedgehogs nor
foxes changed their minds as much as Reverend Bayes says they should
have. But foxes move more in the Bayesian direction than do hybrids and
hedgehogs. And this greater movement is all the more impressive in light
of the fact that the Bayesian updating formula demanded less movement
from foxes than from other groups. Foxes move 59 percent of the pre-
scribed amount, whereas hedgehogs move only 19 percent of the pre-
scribed amount. Indeed, in two regional forecasting exercises, hedgehogs
move their opinions in the opposite direction to that prescribed by Bayes’s
theorem, and nudged up their confidence in their prior point of view after
the unexpected happens. This latter pattern is not just contra-Bayesian; it
is incompatible with all normative theories of belief adjustment.8
https://en.wikipedia.org/wiki/Backfire_effect#Backfire_effect
–