{"id":3800,"date":"2013-05-08T13:54:34","date_gmt":"2013-05-08T12:54:34","guid":{"rendered":"http:\/\/emilkirkegaard.dk\/en\/?p=3800"},"modified":"2013-05-08T13:54:34","modified_gmt":"2013-05-08T12:54:34","slug":"review-of-expert-political-judgement-philip-e-tetlock","status":"publish","type":"post","link":"https:\/\/emilkirkegaard.dk\/en\/2013\/05\/review-of-expert-political-judgement-philip-e-tetlock\/","title":{"rendered":"Review of Expert Political Judgement (Philip E. Tetlock)"},"content":{"rendered":"

[Philip_E._Tetlock]_Expert_Political_Judgment_How(Bookos.org)<\/a><\/p>\n

Very interesting book!<\/p>\n

—-<\/p>\n

Game Theorists. The rivalry between Sherlock Holmes and the evil<\/span><\/p>\n

genius Professor Moriarty illustrates how indeterminacy can arise as a<\/span><\/p>\n

natural by-product of rational agents second-guessing each other. When<\/span><\/p>\n

the two \ufb01rst met, Moriarty was eager, too eager, to display his capacity<\/span><\/p>\n

for interactive thinking by announcing: \u201cAll I have to say has already<\/span><\/p>\n

crossed your mind.\u201d Holmes replied: \u201cThen possibly my answer has<\/span><\/p>\n

crossed yours.\u201d As the plot unfolds, Holmes uses his superior \u201cinterac-<\/span><\/p>\n

tive knowledge\u201d to outmaneuver Moriarty by unexpectedly getting off<\/span><\/p>\n

the train at Canterbury, thwarting Moriarty who had calculated that Paris<\/span><\/p>\n

was Holmes\u2019s rational destination. Convoluted though it is, Moriarty<\/span><\/p>\n

failed to recognize that Holmes had already recognized that Moriarty<\/span><\/p>\n

would deduce what a rational Holmes would do under the circum-<\/span><\/p>\n

stances, and the odds now favored Holmes getting off the train earlier<\/span><\/p>\n

than once planned.23<\/span><\/p>\n

 <\/p>\n

Indeterminacy problems of this sort are the bread and butter of behav-<\/span><\/p>\n

ioral game theory. In the \u201cguess the number\u201d game, for example, con-<\/span><\/p>\n

testants pick a number between 0 and 100, with the goal of making their<\/span><\/p>\n

guess come as close as possible to two-thirds of the average guess of all<\/span><\/p>\n

the contestants.<\/span><\/p>\n

24 In a world of only rational players\u2014who base their<\/span><\/p>\n

guesses on the maximum number of levels of deduction\u2014the equilib-<\/span><\/p>\n

rium is 0. However, in a contest run at Richard Thaler\u2019s prompting by<\/span><\/p>\n

the Financial Times,<\/span><\/p>\n

25 the most popular guesses were 33 (the right guess<\/span><\/p>\n

if everyone else chooses a number at random, producing an average guess<\/span><\/p>\n

of 50) and 22 (the right guess if everyone thinks through the preceding<\/span><\/p>\n

argument and picks 33). Dwindling numbers of respondents carried the<\/span><\/p>\n

deductive logic to the third stage (picking two-thirds of 22) or higher,<\/span><\/p>\n

with a tiny hypereducated group recognizing the logically correct answer<\/span><\/p>\n

to be 0. The average guess was 18.91 and the winning guess, 13, which<\/span><\/p>\n

suggests that, for this newspaper\u2019s readership, a third order of sophisti-<\/span><\/p>\n

cation was roughly optimal.<\/span><\/p>\n

 <\/p>\n

interesting<\/p>\n

 <\/p>\n

–<\/p>\n

 <\/p>\n

Our reluctance to acknowledge unpredictability keeps us looking for<\/span><\/p>\n

predictive cues well beyond the point of diminishing returns. 39 I witnessed<\/span><\/p>\n

a demonstration thirty years ago that pitted the predictive abilities of a<\/span><\/p>\n

classroom of Yale undergraduates against those of a single Norwegian<\/span><\/p>\n

rat. The task was predicting on which side of a T-maze food would ap-<\/span><\/p>\n

pear, with appearances determined\u2014unbeknownst to both the humans<\/span><\/p>\n

and the rat\u2014by a random binomial process (60 percent left and 40 per-<\/span><\/p>\n

cent right). The demonstration replicated the classic studies by Edwards<\/span><\/p>\n

and by Estes: the rat went for the more frequently rewarded side (getting<\/span><\/p>\n

it right roughly 60 percent of the time), whereas the humans looked hard<\/span><\/p>\n

for patterns and wound up choosing the left or the right side in roughly<\/span><\/p>\n

the proportion they were rewarded (getting it right roughly 52 percent of<\/span><\/p>\n

the time). Human performance suffers because we are, deep down, de-<\/span><\/p>\n

terministic thinkers with an aversion to probabilistic strategies that ac-<\/span><\/p>\n

cept the inevitability of error. We insist on looking for order in random<\/span><\/p>\n

sequences. Confronted by the T-maze, we look for subtle patterns like<\/span><\/p>\n

\u201cfood appears in alternating two left\/one right sequences, except after<\/span><\/p>\n

the third cycle when food pops up on the right.\u201d This determination to<\/span><\/p>\n

ferret out order from chaos has served our species well. We are all bene-<\/span><\/p>\n

ficiaries of our great collective successes in the pursuit of deterministic reg-<\/span><\/p>\n

ularities in messy phenomena: agriculture, antibiotics, and countless other<\/span><\/p>\n

inventions that make our comfortable lives possible. But there are occa-<\/span><\/p>\n

sions when the refusal to accept the inevitability of error\u2014to acknowledge<\/span><\/p>\n

that some phenomena are irreducibly probabilistic\u2014can be harmful.<\/span><\/p>\n

 <\/p>\n

indeed, but generally it is wise to not accept the unpredictability hypothesis about some fenomena. many things that were thought unpredictable for centures turned out to be predictable after all, or at least to some degree. i have confidence we will see the same for earthquakes, weather systems and the like in the future as well.<\/p>\n

 <\/p>\n

predictability (and the related determinism) hypothesis are good working hypotheses, even if they turn out to be wrong some times.<\/p>\n

 <\/p>\n

this is what i wrote about years ago on my danish blog here<\/a>. basically, its a 2×2 table:<\/p>\n

 <\/p>\n\n\n\n\n<\/colgroup>\n\n\n\n\n
What we think\/what is true<\/strong><\/td>\nDeterminism<\/strong><\/td>\nIndeterminism<\/strong><\/td>\n<\/tr>\n
Determinism<\/strong><\/td>\nWe keep looking for explanations for fenomena and in over time, we find regularities and explanations.<\/td>\nWe waste time looking for patterns that arent there.<\/td>\n<\/tr>\n
Indeterminism<\/strong><\/td>\nWe dont spend time looking for patterns, but there actually are patterns we that cud use to predict the future, and hence we lose out on possible advances in science.<\/td>\nWe dont waste time looking for patterns that arent there.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

The above is assuming that indeterminism implies total unpredictability. This isnt true, but in the simplified case where were dealing with completely random fenomena and completely predictable fenomena, this is a reasonable way of looking at it. IMO, it is much better to waste time looking for explanations for things that are not orderly (after all), than risk not spotting real patterns in nature.<\/p>\n

 <\/p>\n

Finally, regardless of whether it is rash to abandon the meliorist search<\/span><\/p>\n

for the Holy Grail of good judgment, most of us feel it is. When we weigh<\/span><\/p>\n

the perils of Type I errors (seeking correlates of good judgment that will<\/span><\/p>\n

prove ephemeral) against those of Type II errors (failing to discover<\/span><\/p>\n

durable correlates with lasting value), it does not feel like a close call. We<\/span><\/p>\n

would rather risk anointing lucky fools over ignoring wise counsel. Radi-<\/span><\/p>\n

cal skepticism is too bitter a doctrinal pill for most of us to swallow.<\/span><\/p>\n

 <\/p>\n

exactly<\/p>\n

 <\/p>\n

–<\/p>\n

 <\/p>\n

But betting is one thing, paying up another. Focusing just on reactions<\/span><\/p>\n

to losing reputational bets, \ufb01gure 4.1 shows that neither hedgehogs nor<\/span><\/p>\n

foxes changed their minds as much as Reverend Bayes says they should<\/span><\/p>\n

have. But foxes move more in the Bayesian direction than do hybrids and<\/span><\/p>\n

hedgehogs. And this greater movement is all the more impressive in light<\/span><\/p>\n

of the fact that the Bayesian updating formula demanded less movement<\/span><\/p>\n

from foxes than from other groups. Foxes move 59 percent of the pre-<\/span><\/p>\n

scribed amount, whereas hedgehogs move only 19 percent of the pre-<\/span><\/p>\n

scribed amount. Indeed, in two regional forecasting exercises, hedgehogs<\/span><\/p>\n

move their opinions in the opposite direction to that prescribed by Bayes\u2019s<\/span><\/p>\n

theorem, and nudged up their con\ufb01dence in their prior point of view after<\/span><\/p>\n

the unexpected happens. This latter pattern is not just contra-Bayesian; it<\/span><\/p>\n

is incompatible with all normative theories of belief adjustment.8<\/span><\/p>\n

\n

https:\/\/en.wikipedia.org\/wiki\/Backfire_effect#Backfire_effect<\/a><\/p>\n

\n

–<\/p>\n

\n

\n","protected":false},"excerpt":{"rendered":"

[Philip_E._Tetlock]_Expert_Political_Judgment_How(Bookos.org) Very interesting book! —- Game Theorists. The rivalry between Sherlock Holmes and the evil genius Professor Moriarty illustrates how indeterminacy can arise as a natural by-product of rational agents second-guessing each other. When the two \ufb01rst met, Moriarty was eager, too eager, to display his capacity for interactive thinking by announcing: \u201cAll I have […]<\/p>\n","protected":false},"author":17,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[27,1673],"tags":[1930],"class_list":["post-3800","post","type-post","status-publish","format-standard","hentry","category-epistemology","category-science-philosophy","tag-rationality-philosophy","entry"],"_links":{"self":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/3800","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/comments?post=3800"}],"version-history":[{"count":1,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/3800\/revisions"}],"predecessor-version":[{"id":3802,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/3800\/revisions\/3802"}],"wp:attachment":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/media?parent=3800"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/categories?post=3800"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/tags?post=3800"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}