Review: The Signal and the Noise

The Signal and the Noise Why So Many Predictions Fail – but Some Don’t Nate Silver 544p

 

It is a pretty interesting book especially becus it covers some areas of science not usually covered in popsci (geology, meteorology), and i learned a lot. it is also clearly written and easy to read, which speeds up reading speeds, making the 450ish pages rather quickly to devour. From a learning perspectiv this is awesome as it allows for faster learning. it shud also be mentioned that it has a lot of very useful illustrations which i shared on my social networks while reading it.

 

“Fortunately, Dustin is really cocky, because if he was the kind of person

who was intimidated—if he had listened to those people—it would have ruined

him. He didn’t listen to people. He continued to dig in and swing from his heels

and eventually things turned around for him.”

Pedroia has what John Sanders calls a “major league memory”—which is to

say a short one. He isn’t troubled by a slump, because he is damned sure that

he’s playing the game the right way, and in the long run, that’s what matters.

Indeed, he has very little tolerance for anything that distracts him from doing

his job. This doesn’t make him the most generous human being, but it is ex­

actly what he needs in order to play second base for the Boston Red Sox, and

that’s the only thing that Pedroia cares about.

“Our weaknesses and our strengths are always very intimately connected,”

James said. “Pedroia made strengths out of things that would be weaknesses for

other players.”

 

This sounds like low agreeableness to me. I wonder if Big Five can predict baseball success?

 

-

 

The statistical reality of accuracy isn’t necessarily the governing paradigm

when it comes to commercial weather forecasting. It’s more the perception of

accuracy that adds value in the eyes of the consumer.

For instance, the for-profit weather forecasters rarely predict exactly a

50 percent chance of rain, which might seem wishy-washy and indecisive to

consumers.41 Instead, they’ll flip a coin and round up to 60, or down to 40, even

though this makes the forecasts both less accurate and less honest.42

 

Floehr also uncovered a more flagrant example of fudging the numbers,

something that may be the worst-kept secret in the weather industry. Most com­

mercial weather forecasts are biased, and probably deliberately so. In particu­

lar, they are biased toward forecasting more precipitation than will actually

occur43—what meteorologists call a “wet bias.” The further you get from the

government’s original data, and the more consumer facing the forecasts, the

worse this bias becomes. Forecasts “add value” by subtracting accuracy.

 

thats interesting. never heard of this.

 

-

 

This logic is a little circular. TV weathermen say they aren’t bothering to

make accurate forecasts because they figure the public won’t believe them any­

way. But the public shouldn t believe them, because the forecasts aren’t accurate.

This becomes a more serious problem when there is something urgent—

something like Hurricane Katrina. Lots of Americans get their weather infor­

mation from local sources49 rather than directly from the Hurricane Center, so

they will still be relying on the goofball on Channel 7 to provide them with

accurate information. If there is a mutual distrust between the weather fore­

caster and the public, the public may not listen when they need to most.

 

Nicely illustrating for importance of honesty in reporting data, even on local TV.

 

-

 

In fact, the actual value for GDP fell outside the economists’ prediction

interval six times in eighteen years, or fully one-third of the time. Another

study,18 which ran these numbers back to the beginnings of the Survey of Pro­

fessional Forecasters in 1968, found even worse results: the actual figure for

GDP fell outside the prediction interval almost h a l f the time. There is almost

no chance19 that the economists have simply been unlucky; they fundamentally

overstate the reliability of their predictions.

 

In reality, when a group of economists give you their GDP forecast, the

true 90 percent prediction interval—based on how these forecasts have actually

performed20 and not on how accurate the economists claim them to be—spans

about 6.4 points of GDP (equivalent to a margin of error of plus or minus 3.2

percent).*

 

When you hear on the news that GDP will grow by 2.5 percent next year,

that means it could quite easily grow at a spectacular rate of 5.7 percent instead.

Or it could fall by 0.7 percent—a fairly serious recession. Economists haven’t

been able to do any better than that, and there isn’t much evidence that their

forecasts are improving. The old joke about economists’ having called nine out

of the last six recessions correctly has some truth to it; one actual statistic is that

in the 1990s, economists predicted only 2 of the 60 recessions around the world

a year ahead of time.21

 

and this is why we cant have nice things, i mean macroeconomics

 

-

 

I have no idea whether I was really a good player at the very outset. But the

bar set by the competition was low, and my statistical background gave me an

advantage. Poker is sometimes perceived to be a highly psychological game, a

battle of wills in which opponents seek to make perfect reads on one another by

staring into one another’s souls, looking for “tells” that reliably betray the con­

tents of the other hands. There is a little bit of this in poker, especially at the

higher limits, but not nearly as much as you’d think. (The psychological factors

in poker come mostly in the form of self-discipline.) Instead, poker is an incred­

ibly mathematical game that depends on making probabilistic judgments amid

uncertainty, the same skills that are important in any type of prediction.

 

The obvious idea is to program computers to play poker for u online. If they play against bad humans, they shud bring in a steady flow of cash for almost free.

 

-

 

 

“Fortunately, Dustin is really cocky, because if he was the kind of person

who was intimidated—if he had listened to those people—it would have ruined

him. He didn’t listen to people. He continued to dig in and swing from his heels

and eventually things turned around for him.”

Pedroia has what John Sanders calls a “major league memory”—which is to

say a short one. He isn’t troubled by a slump, because he is damned sure that

he’s playing the game the right way, and in the long run, that’s what matters.

Indeed, he has very little tolerance for anything that distracts him from doing

his job. This doesn’t make him the most generous human being, but it is ex­

actly what he needs in order to play second base for the Boston Red Sox, and

that’s the only thing that Pedroia cares about.

“Our weaknesses and our strengths are always very intimately connected,”

James said. “Pedroia made strengths out of things that would be weaknesses for

other players.”

This sounds like low agreeableness to me. I wonder if Big Five can predict baseball success?

-

The statistical reality of accuracy isn’t necessarily the governing paradigm

when it comes to commercial weather forecasting. It’s more the perception of

accuracy that adds value in the eyes of the consumer.

For instance, the for-profit weather forecasters rarely predict exactly a

50 percent chance of rain, which might seem wishy-washy and indecisive to

consumers.41 Instead, they’ll flip a coin and round up to 60, or down to 40, even

though this makes the forecasts both less accurate and less honest.42

Floehr also uncovered a more flagrant example of fudging the numbers,

something that may be the worst-kept secret in the weather industry. Most com­

mercial weather forecasts are biased, and probably deliberately so. In particu­

lar, they are biased toward forecasting more precipitation than will actually

occur43—what meteorologists call a “wet bias.” The further you get from the

government’s original data, and the more consumer facing the forecasts, the

worse this bias becomes. Forecasts “add value” by subtracting accuracy.

thats interesting. never heard of this.

-

This logic is a little circular. TV weathermen say they aren’t bothering to

make accurate forecasts because they figure the public won’t believe them any­

way. But the public shouldn t believe them, because the forecasts aren’t accurate.

This becomes a more serious problem when there is something urgent—

something like Hurricane Katrina. Lots of Americans get their weather infor­

mation from local sources49 rather than directly from the Hurricane Center, so

they will still be relying on the goofball on Channel 7 to provide them with

accurate information. If there is a mutual distrust between the weather fore­

caster and the public, the public may not listen when they need to most.

Nicely illustrating for importance of honesty in reporting data, even on local TV.

-

In fact, the actual value for GDP fell outside the economists’ prediction

interval six times in eighteen years, or fully one-third of the time. Another

study,18 which ran these numbers back to the beginnings of the Survey of Pro­

fessional Forecasters in 1968, found even worse results: the actual figure for

GDP fell outside the prediction interval almost h a l f the time. There is almost

no chance19 that the economists have simply been unlucky; they fundamentally

overstate the reliability of their predictions.

In reality, when a group of economists give you their GDP forecast, the

true 90 percent prediction interval—based on how these forecasts have actually

performed20 and not on how accurate the economists claim them to be—spans

about 6.4 points of GDP (equivalent to a margin of error of plus or minus 3.2

percent).*

When you hear on the news that GDP will grow by 2.5 percent next year,

that means it could quite easily grow at a spectacular rate of 5.7 percent instead.

Or it could fall by 0.7 percent—a fairly serious recession. Economists haven’t

been able to do any better than that, and there isn’t much evidence that their

forecasts are improving. The old joke about economists’ having called nine out

of the last six recessions correctly has some truth to it; one actual statistic is that

in the 1990s, economists predicted only 2 of the 60 recessions around the world

a year ahead of time.21

and this is why we cant have nice things, i mean macroeconomics

-

I have no idea whether I was really a good player at the very outset. But the

bar set by the competition was low, and my statistical background gave me an

advantage. Poker is sometimes perceived to be a highly psychological game, a

battle of wills in which opponents seek to make perfect reads on one another by

staring into one another’s souls, looking for “tells” that reliably betray the con­

tents of the other hands. There is a little bit of this in poker, especially at the

higher limits, but not nearly as much as you’d think. (The psychological factors

in poker come mostly in the form of self-discipline.) Instead, poker is an incred­

ibly mathematical game that depends on making probabilistic judgments amid

uncertainty, the same skills that are important in any type of prediction.

The obvious idea is to program computers to play poker for u online. If they play against bad humans, they shud bring in a steady flow of cash for almost free.

-

Why Open Access is a moral imperative

i hav riten about it befor, but here is som mor i came acros.
Found via:
www.techdirt.com/articles/20130117/03040821712/scientist-explains-why-putting-research-behind-paywall-is-immoral.shtml

For general public:
www.guardian.co.uk/science/blog/2013/jan/17/open-access-publishing-science-paywall-immoral
www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=417576&c=1
www.guardian.co.uk/science/2012/jul/15/free-access-british-scientific-research

Background material for the curious:
Blog series:
svpow.com/2009/06/01/choosing-a-journal-for-the-neck-posture-paper-why-open-access-is-important/
svpow.com/2011/09/29/researchers-stop-doing-free-work-for-non-open-journals/
svpow.com/2011/10/17/collateral-damage-of-the-non-open-reviewing-boycott/
svpow.com/2012/01/30/what-is-a-private-sector-research-work/
svpow.com/2012/03/30/my-rcuk-submission/
svpow.com/2012/05/17/see-this-is-why-publishers-irritate-me-so-much/
svpow.com/2012/05/18/publishers-versus-everyone/
svpow.com/2012/10/16/publish-means-make-public-paywalls-are-the-opposite-of-publishing/

Other:
drvector.blogspot.dk/2007/02/reason-317-to-be-depressed-journal.html
www.scottaaronson.com/writings/journal.html
www.michaeleisen.org/blog/?p=911

Paper: “Positive” Results Increase Down the Hierarchy of the Sciences

www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0010068

 

 

The hypothesised Hierarchy of the Sciences (henceforth HoS) is

reflected inmany social and organizational features of academic life.

When 222 scholars rated their perception of similarity between

academic disciplines, results showed a clustering along three main

dimensions: a ‘‘hard/soft’’ dimension, which roughly corresponded

to the HoS; a ‘‘pure/applied’’ dimension, which reflected the

orientation of the discipline towards practical application; and a

‘‘life/non-life’’ dimension [13]. These dimensions have been vali-

dated by many subsequent studies, which compared disciplines by

parameters including: average publication rate of scholars, level of

social connectedness, level of job satisfaction, professional commit-

ment, approaches to learning, goals of academic departments,

professional duties of department heads, financial reward structures

of academic departments, and even response rates to survey

questionnaires [14,15,16,17].

 

refs are:

13. Biglan A (1973) Characteristics of subject matter in different academic areas.

Journal of Applied Psychology 57: 195–203.

 

14. Smart JC, Elton CF (1982) Validation of the Biglan model. Research in Higher

Education 17: 213–229.

 

15. Malaney GD (1986) Differentiation in graduate-education. Research in Higher

Education 25: 82–96.

 

16. Stoecker JL (1993) The Biglan classification revisited. Research in Higher

Education 34: 451–464.

 

17. Laird TFN, Shoup R, Kuh GD, Schwarz MJ (2008) The effects of discipline on

deep approaches to student learning and college outcomes. Research in Higher

Education 49: 469–494.

 

Numerous studies have taken a direct approach, and have

attempted to compare the hardness of two or more disciplines,

usually psychology or sociology against one or more of the natural

sciences. These studies used a variety of proxy measures including:

ratio of theories to laws in introductory textbooks, number of

colleagues acknowledged in papers, publication cost of interrupt-

ing academic career for one year, proportion of under 35 s who

received above-average citations, concentration of citations in the

literature, rate of pauses in lectures given to undergraduates,

immediacy of citations, anticipation of one’s work by colleagues,

average age when receiving the Nobel prize, fraction of journals’

space occupied by graphs (called Fractional Graph Area, or FGA),

and others [17,18]. According to a recent review, some of these

measures are correlated to one-another and to the HoS [2]. One

parameter, FGA, even appears to capture the relative hardness of

sub-disciplines: in psychology, FGA is higher in journals rated as

‘‘harder’’ by psychologists, and also in journals specialised in

animal behaviour rather than human behaviour [19,20,21].

 

refs are:

19. Best LA, Smith LD, Stubbs DA (2001) Graph use in psychology and other

sciences. Behavioural Processes 54: 155–165.

 

20. Kubina RM, Kostewicz DE, Datchuk SM (2008) An initial survey of fractional

graph and table area in behavioral journals. Behavior Analyst 31: 61–66.

 

21. Smith LD, Best LA, Stubbs DA, Johnston J, Archibald AB (2000) Scientific

graphs and the hierarchy of the sciences: A Latourian survey of inscription

practices. Social Studies of Science 30: 73–94.

 

a very interesting metascience paper! refs are also interesting

 

-

 

 

Some random things that i read recently+thoughts

www.chicagotribune.com/sns-rt-brazil-prisonreadingl2e8hp75l-20120625,0,7694158.story

This is an interesting idea.

-

news.yahoo.com/math-anxiety-school-scientists-too-190128180–abc-news-tech.html

www.pnas.org/content/early/2012/06/22/1205259109

Not surprised. Surely, there is a similar aversion among filosofy students at my university to logic, since its the formal near-equivalent of math. And when i change to linguistics this fall, i expect to see a similar aversion to formal linguistics (say, generative grammar).

I wonder, is there an opposite effect, an aversion to words in math depts? Word/language anxiety?

-

maggiemcneill.wordpress.com/

“The Honest Courtesan

Frank commentary from a retired call girl”

An anti-neofeminism blog. Deals alot with dumb politicians and sex and sex trade etc. Added to my feed. Generally, i find hookers interesting. Unfortunately, i dont know any afaik.

-

medicalhypotheses.blogspot.co.uk/

en.wikipedia.org/wiki/Medical_Hypotheses

 “Medical Hypotheses is a medical journal published by Elsevier. It was originally intended as a forum for unconventional ideas without the traditional filter of scientific peer review, “so long as (the ideas) are coherent and clearly expressed” in order to “foster the diversity and debate upon which the scientific process thrives.”[1] Medical Hypotheses was the only Elsevier journal that did not send submitted papers to other scientists for review.[2] Articles were chosen instead by the journal’s editor-in-chief based on whether he considered the submitted work interesting and important. The journal’s policy placed full responsibility for the integrity, precision and accuracy of publications on the authors, rather than peer reviewers or the editor.[3]

The journal’s lack of peer review[4] and publication of ideas that are considered clear pseudoscience,[5] particularly AIDS denialism,[6] attracted considerable criticism, including calls to remove it from PubMed, the prestigious United States National Library of Medicine online journal database.[5] Following the AIDS papers controversy, Elsevier forced a change in the journal’s leadership. In June 2010, Elsevier announced that “Submitted manuscripts will be reviewed by the Editor and external reviewers to ensure their scientific merit”, suggesting that peer review is now in place.[7]

Too bad. Pre-print peer-review is not that good an idea (perhaps not a good idea at all), as i have previously posted about, twice.

The author of the blog is quite interesting, even if he is a crazy xtian (see his other blog).

I have been reading alot of posts from his blog:

medicalhypotheses.blogspot.co.uk/2010/02/why-are-women-so-intelligent.html

medicalhypotheses.blogspot.co.uk/2009/05/do-elite-us-colleges-choose-personality.html

medicalhypotheses.blogspot.co.uk/2009/02/why-are-modern-scientists-so-dull.html

This one was especially interesting. Also, R. Lynn’s general model of achievement seems about right, tho not quite: IQ × Conscientiousness × opportunity = Achievement. I propose this one instead:

The reasoning being: 1) It is intelligence and not IQ that is important. IQ is only important in as far as it correlates with g. Thus, substitute those two. 2) Similarly for conscientiousness. It is only relevant in as far as it predicts amount of work/effort. One can increase workload without increasing C score, and the result is more work done = achievement. Thus, substitute C for work. 3) Some things may have a minimum threshold of g. I suppose it is impossible to teach, say, linear algebra til someone who is at g = 80. If there is no threshold within the domain, one can just set it to 0. 4) An even more general model is perhaps this:

Note the added WorkingSpeed. With matters related to intelligence, this is not relevant as ability is pretty much the same as speed, but there are some contexts where those are not the same. There is however also an issue with non-linear effects. One can get around this by having all the factors as functions like this:

I will be reading these posts in the future:

medicalhypotheses.blogspot.co.uk/2009/07/replacing-education-with-psychometrics.html

medicalhypotheses.blogspot.co.uk/2008/07/mavericks-versus-team-players.html

medicalhypotheses.blogspot.co.uk/2009/08/reliable-but-dumb-or-smart-but-slapdash.html

medicalhypotheses.blogspot.co.uk/2009/10/truthfulness-in-science-should-be-iron.html

and perhaps some more.