{"id":3786,"date":"2013-04-23T00:53:42","date_gmt":"2013-04-22T23:53:42","guid":{"rendered":"http:\/\/emilkirkegaard.dk\/en\/?p=3786"},"modified":"2013-04-23T00:53:42","modified_gmt":"2013-04-22T23:53:42","slug":"review-the-signal-and-the-noise","status":"publish","type":"post","link":"https:\/\/emilkirkegaard.dk\/en\/2013\/04\/review-the-signal-and-the-noise\/","title":{"rendered":"Review: The Signal and the Noise"},"content":{"rendered":"

The Signal and the Noise Why So Many Predictions Fail – but Some Don\u2019t Nate Silver 544p<\/a><\/p>\n

 <\/p>\n

It is a pretty interesting book especially becus it covers some areas of science not usually covered in popsci (geology, meteorology), and i learned a lot. it is also clearly written and easy to read, which speeds up reading speeds, making the 450ish pages rather quickly to devour. From a learning perspectiv this is awesome as it allows for faster learning. it shud also be mentioned that it has a lot of very useful illustrations which i shared on my social networks while reading it.<\/p>\n

 <\/p>\n

—<\/p>\n

\u201cFortunately, Dustin is really cocky, because if he was the kind of person <\/span><\/p>\n

who was intimidated\u2014if he had listened to those people\u2014it would have ruined <\/span><\/p>\n

him. He didn\u2019t listen to people. He continued to dig in and swing from his heels <\/span><\/p>\n

and eventually things turned around for him.\u201d<\/span><\/p>\n

Pedroia has what John Sanders calls a \u201cmajor league memory\u201d\u2014which is to <\/span><\/p>\n

say a short one. He isn’t troubled by a slump, because he is damned sure that <\/span><\/p>\n

he\u2019s playing the game the right way, and in the long run, that\u2019s what matters. <\/span><\/p>\n

Indeed, he has very little tolerance for anything that distracts him from doing <\/span><\/p>\n

his job. This doesn\u2019t make him the most generous human being, but it is ex\u00ad<\/span><\/p>\n

actly what he needs in order to play second base for the Boston Red Sox, and <\/span><\/p>\n

that\u2019s the only thing that Pedroia cares about.<\/span><\/p>\n

\u201cOur weaknesses and our strengths are always very intimately connected,\u201d <\/span><\/p>\n

James said. \u201cPedroia made strengths out of things that would be weaknesses for <\/span><\/p>\n

other players.\u201d<\/span><\/p>\n

 <\/p>\n

This sounds like low agreeableness to me. I wonder if Big Five can predict baseball success?<\/p>\n

 <\/p>\n

–<\/p>\n

 <\/p>\n

The statistical reality of accuracy isn\u2019t necessarily the governing paradigm <\/span><\/p>\n

when it comes to commercial weather forecasting. It\u2019s more the perception of <\/span><\/p>\n

accuracy that adds value in the eyes of the consumer.<\/span><\/p>\n

For instance, the for-profit weather forecasters rarely predict exactly a <\/span><\/p>\n

50 percent chance of rain, which might seem wishy-washy and indecisive to <\/span><\/p>\n

consumers.41 Instead, they\u2019ll flip a coin and round up to 60, or down to 40, even <\/span><\/p>\n

though this makes the forecasts both less accurate and less honest.42<\/span><\/p>\n

 <\/p>\n

Floehr also uncovered a more flagrant example of fudging the numbers,<\/span><\/p>\n

something that may be the worst-kept secret in the weather industry. Most com\u00ad<\/span><\/p>\n

mercial weather forecasts are biased, and probably deliberately so. In particu\u00ad<\/span><\/p>\n

lar, they are biased toward forecasting more precipitation than will actually <\/span><\/p>\n

occur43\u2014what meteorologists call a \u201cwet bias.\u201d The further you get from the <\/span><\/p>\n

government\u2019s original data, and the more consumer facing the forecasts, the <\/span><\/p>\n

worse this bias becomes. Forecasts \u201cadd value\u201d by subtracting accuracy.<\/span><\/p>\n

 <\/p>\n

thats interesting. never heard of this.<\/p>\n

 <\/p>\n

–<\/p>\n

 <\/p>\n

This logic is a little circular. TV weathermen say they aren\u2019t bothering to <\/span><\/p>\n

make accurate forecasts because they figure the public won’t believe them any\u00ad<\/span><\/p>\n

way. But the public shouldn t believe them, because the forecasts aren\u2019t accurate.<\/span><\/p>\n

This becomes a more serious problem when there is something urgent\u2014 <\/span><\/p>\n

something like Hurricane Katrina. Lots of Americans get their weather infor\u00ad<\/span><\/p>\n

mation from local sources49 rather than directly from the Hurricane Center, so <\/span><\/p>\n

they will still be relying on the goofball on Channel 7 to provide them with <\/span><\/p>\n

accurate information. If there is a mutual distrust between the weather fore\u00ad<\/span><\/p>\n

caster and the public, the public may not listen when they need to most.<\/span><\/p>\n

 <\/p>\n

Nicely illustrating for importance of honesty in reporting data, even on local TV.<\/p>\n

 <\/p>\n

–<\/p>\n

 <\/p>\n

In fact, the actual value for GDP fell outside the economists\u2019 prediction <\/span><\/p>\n

interval six times in eighteen years, or fully one-third of the time. Another <\/span><\/p>\n

study,18 which ran these numbers back to the beginnings of the Survey of Pro\u00ad<\/span><\/p>\n

fessional Forecasters in 1968, found even worse results: the actual figure for <\/span><\/p>\n

GDP fell outside the prediction interval almost h a l f the time. There is almost <\/span><\/p>\n

no chance19 that the economists have simply been unlucky; they fundamentally <\/span><\/p>\n

overstate the reliability of their predictions.<\/span><\/p>\n

 <\/p>\n

In reality, when a group of economists give you their GDP forecast, the <\/span><\/p>\n

true 90 percent prediction interval\u2014based on how these forecasts have actually <\/span><\/p>\n

performed20 and not on how accurate the economists claim them to be\u2014spans<\/span><\/p>\n

about 6.4 points of GDP (equivalent to a margin of error of plus or minus 3.2 <\/span><\/p>\n

percent).*<\/span><\/p>\n

 <\/p>\n

When you hear on the news that GDP will grow by 2.5 percent next year, <\/span><\/p>\n

that means it could quite easily grow at a spectacular rate of 5.7 percent instead. <\/span><\/p>\n

Or it could fall by 0.7 percent\u2014a fairly serious recession. Economists haven\u2019t <\/span><\/p>\n

been able to do any better than that, and there isn\u2019t much evidence that their <\/span><\/p>\n

forecasts are improving. The old joke about economists\u2019 having called nine out <\/span><\/p>\n

of the last six recessions correctly has some truth to it; one actual statistic is that <\/span><\/p>\n

in the 1990s, economists predicted only 2 of the 60 recessions around the world <\/span><\/p>\n

a year ahead of time.21<\/span><\/p>\n

 <\/p>\n

and this is why we cant have nice things, i mean macroeconomics<\/p>\n

 <\/p>\n

–<\/p>\n

 <\/p>\n

I have no idea whether I was really a good player at the very outset. But the <\/span><\/p>\n

bar set by the competition was low, and my statistical background gave me an <\/span><\/p>\n

advantage. Poker is sometimes perceived to be a highly psychological game, a <\/span><\/p>\n

battle of wills in which opponents seek to make perfect reads on one another by <\/span><\/p>\n

staring into one another\u2019s souls, looking for \u201ctells\u201d that reliably betray the con\u00ad<\/span><\/p>\n

tents of the other hands. There is a little bit of this in poker, especially at the <\/span><\/p>\n

higher limits, but not nearly as much as you\u2019d think. (The psychological factors <\/span><\/p>\n

in poker come mostly in the form of self-discipline.) Instead, poker is an incred\u00ad<\/span><\/p>\n

ibly mathematical game that depends on making probabilistic judgments amid <\/span><\/p>\n

uncertainty, the same skills that are important in any type of prediction.<\/span><\/p>\n

 <\/p>\n

The obvious idea is to program computers to play poker for u online. If they play against bad humans, they shud bring in a steady flow of cash for almost free.<\/p>\n

 <\/p>\n

–<\/p>\n

 <\/p>\n

 <\/p>\n

\u201cFortunately, Dustin is really cocky, because if he was the kind of person <\/span><\/p>\n

who was intimidated\u2014if he had listened to those people\u2014it would have ruined <\/span><\/p>\n

him. He didn\u2019t listen to people. He continued to dig in and swing from his heels <\/span><\/p>\n

and eventually things turned around for him.\u201d<\/span><\/p>\n

Pedroia has what John Sanders calls a \u201cmajor league memory\u201d\u2014which is to <\/span><\/p>\n

say a short one. He isn’t troubled by a slump, because he is damned sure that <\/span><\/p>\n

he\u2019s playing the game the right way, and in the long run, that\u2019s what matters. <\/span><\/p>\n

Indeed, he has very little tolerance for anything that distracts him from doing <\/span><\/p>\n

his job. This doesn\u2019t make him the most generous human being, but it is ex\u00ad<\/span><\/p>\n

actly what he needs in order to play second base for the Boston Red Sox, and <\/span><\/p>\n

that\u2019s the only thing that Pedroia cares about.<\/span><\/p>\n

\u201cOur weaknesses and our strengths are always very intimately connected,\u201d <\/span><\/p>\n

James said. \u201cPedroia made strengths out of things that would be weaknesses for <\/span><\/p>\n

other players.\u201d<\/span><\/p>\n

\n

This sounds like low agreeableness to me. I wonder if Big Five can predict baseball success?<\/p>\n

\n

–<\/p>\n

\n

The statistical reality of accuracy isn\u2019t necessarily the governing paradigm <\/span><\/p>\n

when it comes to commercial weather forecasting. It\u2019s more the perception of <\/span><\/p>\n

accuracy that adds value in the eyes of the consumer.<\/span><\/p>\n

For instance, the for-profit weather forecasters rarely predict exactly a <\/span><\/p>\n

50 percent chance of rain, which might seem wishy-washy and indecisive to <\/span><\/p>\n

consumers.41 Instead, they\u2019ll flip a coin and round up to 60, or down to 40, even <\/span><\/p>\n

though this makes the forecasts both less accurate and less honest.42<\/span><\/p>\n

\n

Floehr also uncovered a more flagrant example of fudging the numbers,<\/span><\/p>\n

something that may be the worst-kept secret in the weather industry. Most com\u00ad<\/span><\/p>\n

mercial weather forecasts are biased, and probably deliberately so. In particu\u00ad<\/span><\/p>\n

lar, they are biased toward forecasting more precipitation than will actually <\/span><\/p>\n

occur43\u2014what meteorologists call a \u201cwet bias.\u201d The further you get from the <\/span><\/p>\n

government\u2019s original data, and the more consumer facing the forecasts, the <\/span><\/p>\n

worse this bias becomes. Forecasts \u201cadd value\u201d by subtracting accuracy.<\/span><\/p>\n

\n

thats interesting. never heard of this.<\/p>\n

\n

–<\/p>\n

\n

This logic is a little circular. TV weathermen say they aren\u2019t bothering to <\/span><\/p>\n

make accurate forecasts because they figure the public won’t believe them any\u00ad<\/span><\/p>\n

way. But the public shouldn t believe them, because the forecasts aren\u2019t accurate.<\/span><\/p>\n

This becomes a more serious problem when there is something urgent\u2014 <\/span><\/p>\n

something like Hurricane Katrina. Lots of Americans get their weather infor\u00ad<\/span><\/p>\n

mation from local sources49 rather than directly from the Hurricane Center, so <\/span><\/p>\n

they will still be relying on the goofball on Channel 7 to provide them with <\/span><\/p>\n

accurate information. If there is a mutual distrust between the weather fore\u00ad<\/span><\/p>\n

caster and the public, the public may not listen when they need to most.<\/span><\/p>\n

\n

Nicely illustrating for importance of honesty in reporting data, even on local TV.<\/p>\n

\n

–<\/p>\n

\n

In fact, the actual value for GDP fell outside the economists\u2019 prediction <\/span><\/p>\n

interval six times in eighteen years, or fully one-third of the time. Another <\/span><\/p>\n

study,18 which ran these numbers back to the beginnings of the Survey of Pro\u00ad<\/span><\/p>\n

fessional Forecasters in 1968, found even worse results: the actual figure for <\/span><\/p>\n

GDP fell outside the prediction interval almost h a l f the time. There is almost <\/span><\/p>\n

no chance19 that the economists have simply been unlucky; they fundamentally <\/span><\/p>\n

overstate the reliability of their predictions.<\/span><\/p>\n

\n

In reality, when a group of economists give you their GDP forecast, the <\/span><\/p>\n

true 90 percent prediction interval\u2014based on how these forecasts have actually <\/span><\/p>\n

performed20 and not on how accurate the economists claim them to be\u2014spans<\/span><\/p>\n

about 6.4 points of GDP (equivalent to a margin of error of plus or minus 3.2 <\/span><\/p>\n

percent).*<\/span><\/p>\n

\n

When you hear on the news that GDP will grow by 2.5 percent next year, <\/span><\/p>\n

that means it could quite easily grow at a spectacular rate of 5.7 percent instead. <\/span><\/p>\n

Or it could fall by 0.7 percent\u2014a fairly serious recession. Economists haven\u2019t <\/span><\/p>\n

been able to do any better than that, and there isn\u2019t much evidence that their <\/span><\/p>\n

forecasts are improving. The old joke about economists\u2019 having called nine out <\/span><\/p>\n

of the last six recessions correctly has some truth to it; one actual statistic is that <\/span><\/p>\n

in the 1990s, economists predicted only 2 of the 60 recessions around the world <\/span><\/p>\n

a year ahead of time.21<\/span><\/p>\n

\n

and this is why we cant have nice things, i mean macroeconomics<\/p>\n

\n

–<\/p>\n

\n

I have no idea whether I was really a good player at the very outset. But the <\/span><\/p>\n

bar set by the competition was low, and my statistical background gave me an <\/span><\/p>\n

advantage. Poker is sometimes perceived to be a highly psychological game, a <\/span><\/p>\n

battle of wills in which opponents seek to make perfect reads on one another by <\/span><\/p>\n

staring into one another\u2019s souls, looking for \u201ctells\u201d that reliably betray the con\u00ad<\/span><\/p>\n

tents of the other hands. There is a little bit of this in poker, especially at the <\/span><\/p>\n

higher limits, but not nearly as much as you\u2019d think. (The psychological factors <\/span><\/p>\n

in poker come mostly in the form of self-discipline.) Instead, poker is an incred\u00ad<\/span><\/p>\n

ibly mathematical game that depends on making probabilistic judgments amid <\/span><\/p>\n

uncertainty, the same skills that are important in any type of prediction.<\/span><\/p>\n

\n

The obvious idea is to program computers to play poker for u online. If they play against bad humans, they shud bring in a steady flow of cash for almost free.<\/p>\n

\n

–<\/p>\n

\n

\n","protected":false},"excerpt":{"rendered":"

The Signal and the Noise Why So Many Predictions Fail – but Some Don\u2019t Nate Silver 544p   It is a pretty interesting book especially becus it covers some areas of science not usually covered in popsci (geology, meteorology), and i learned a lot. it is also clearly written and easy to read, which speeds […]<\/p>\n","protected":false},"author":17,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1898,27,2049,1107,1921],"tags":[],"class_list":["post-3786","post","type-post","status-publish","format-standard","hentry","category-economics","category-epistemology","category-metascience","category-science","category-sociology","entry"],"_links":{"self":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/3786","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/comments?post=3786"}],"version-history":[{"count":1,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/3786\/revisions"}],"predecessor-version":[{"id":3788,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/3786\/revisions\/3788"}],"wp:attachment":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/media?parent=3786"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/categories?post=3786"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/tags?post=3786"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}