{"id":8629,"date":"2020-05-07T03:10:05","date_gmt":"2020-05-07T02:10:05","guid":{"rendered":"https:\/\/emilkirkegaard.dk\/en\/?p=8629"},"modified":"2020-05-07T03:10:05","modified_gmt":"2020-05-07T02:10:05","slug":"reverse-publication-bias-a-collection","status":"publish","type":"post","link":"https:\/\/emilkirkegaard.dk\/en\/2020\/05\/reverse-publication-bias-a-collection\/","title":{"rendered":"Reverse publication bias: a collection"},"content":{"rendered":"<p><a href=\"https:\/\/emilkirkegaard.dk\/en\/?p=8322\"><em>See also Sesardi\u0107&#8217;s conjecture, a related idea.<\/em><\/a><\/p>\n<p><a href=\"https:\/\/en.wikipedia.org\/wiki\/Publication_bias\">Publication bias<\/a> as normally considered is really <em>positive <\/em>publication bias, i.e., the bias is away from zero, towards finding larger than reality results. There is another form, however, more rare, called reverse publication bias, or negative publication bias, where published results are biased towards the null. This pattern results from researchers&#8217; own biases towards having something noteworthy to report, wanting to publish it, and getting through peer and editorial review. All of these things are related to political ideology of the researchers and their academic environment (self-censorship). <a href=\"https:\/\/www.tandfonline.com\/doi\/abs\/10.1080\/1047840X.2020.1722600?journalCode=hpli20\">Thus, due to the massive political ideology skew in social science<\/a>, we expect:<\/p>\n<ol>\n<li>Positive publication bias for left-wing friendly and results in general<\/li>\n<li>Negative\/reverse publication bias for results that left-wing unfriendly results<\/li>\n<\/ol>\n<p>Below I list the examples I know of.<\/p>\n<h3>GPA and intelligence<\/h3>\n<ul>\n<li>\n<div class=\"gs_citr\" tabindex=\"0\">Roth, B., Becker, N., Romeyke, S., Sch\u00e4fer, S., Domnick, F., &amp; Spinath, F. M. (2015). <a href=\"https:\/\/www.sciencedirect.com\/science\/article\/abs\/pii\/S0160289615001269\">Intelligence and school grades: A meta-analysis<\/a>. <i>Intelligence<\/i>, <i>53<\/i>, 118-137.<\/div>\n<\/li>\n<\/ul>\n<div class=\"oceanwp-oembed-wrap clr\">\n<blockquote class=\"twitter-tweet\" data-width=\"550\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">Reverse publication bias: when scientists want to find smaller effects. (IQ and GPA) <a href=\"https:\/\/t.co\/SyWhjMcYFf\">https:\/\/t.co\/SyWhjMcYFf<\/a> <a href=\"https:\/\/t.co\/77Yw3OdtyR\">pic.twitter.com\/77Yw3OdtyR<\/a><\/p>\n<p>&mdash; Emil O W Kirkegaard (@KirkegaardEmil) <a href=\"https:\/\/twitter.com\/KirkegaardEmil\/status\/711355039248416768?ref_src=twsrc%5Etfw\">March 20, 2016<\/a><\/p><\/blockquote>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/div>\n<h3>Sex difference in spatial ability<\/h3>\n<p>This study was still unpublished when I asked the Jakob about it Apr 11, 2019.<\/p>\n<div class=\"oceanwp-oembed-wrap clr\">\n<blockquote class=\"twitter-tweet\" data-width=\"550\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">Jakob Pietschnig: Reverse publication bias for sex difference in spatial ability. Psychologists publish as if trying to hide the male advantage. <a href=\"https:\/\/twitter.com\/hashtag\/ISIR2018?src=hash&amp;ref_src=twsrc%5Etfw\">#ISIR2018<\/a> <a href=\"https:\/\/t.co\/lxmuUyXV17\">https:\/\/t.co\/lxmuUyXV17<\/a> <a href=\"https:\/\/t.co\/n4h2qm5NE6\">pic.twitter.com\/n4h2qm5NE6<\/a><\/p>\n<p>&mdash; Emil O W Kirkegaard (@KirkegaardEmil) <a href=\"https:\/\/twitter.com\/KirkegaardEmil\/status\/1018494702977474561?ref_src=twsrc%5Etfw\">July 15, 2018<\/a><\/p><\/blockquote>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/div>\n<h3>Race difference in personality (OCEAN)<\/h3>\n<ul>\n<li>\n<div class=\"gs_citr\" tabindex=\"0\">Tate, B. W., &amp; Mcdaniel, M. A. (2008). <a href=\"https:\/\/www.researchgate.net\/publication\/237215311_Race_Differences_in_Personality_An_Evaluation_of_Moderators_and_Publication_Bias\">Race differences in personality: An evaluation of moderators and publication bias<\/a>. Preprint from before preprints were cool.<\/div>\n<\/li>\n<\/ul>\n<div class=\"oceanwp-oembed-wrap clr\">\n<blockquote class=\"twitter-tweet\" data-width=\"550\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">More reverse publication bias: race differences in personality. Altho these gaps are fairly trivial for US black-white on self-reported OCEAN, there&#39;s still evidence of suppression. Seems in general that group difference research often shows reverse bias.<a href=\"https:\/\/t.co\/0O6IwUPtD3\">https:\/\/t.co\/0O6IwUPtD3<\/a> <a href=\"https:\/\/t.co\/Vc1DunY3py\">pic.twitter.com\/Vc1DunY3py<\/a><\/p>\n<p>&mdash; Emil O W Kirkegaard (@KirkegaardEmil) <a href=\"https:\/\/twitter.com\/KirkegaardEmil\/status\/1096987143019335681?ref_src=twsrc%5Etfw\">February 17, 2019<\/a><\/p><\/blockquote>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/div>\n<h3>Other<\/h3>\n<p>One can find recent discussions by searching Twitter: <a href=\"https:\/\/twitter.com\/search?q=reverse%20publication%20bias&amp;f=live\">reverse publication bias<\/a> or more strict with quotes &#8220;reverse publication bias&#8221;.<\/p>\n<p>A bit of a side-interest is when people are not trying to suppress some finding, but just want to avoid the result of a statistical test of an assumption being violated. This can be from testing means in post-randomization groups (these are hopefully all false positives that should happen 5% of the time), or testing for model assumptions (e.g. normal distribution). There&#8217;s a lot of these kind of meta papers, two examples:<\/p>\n<ul>\n<li>\n<div class=\"gs_citr\" tabindex=\"0\">Chuard, P. J., Vrt\u00edlek, M., Head, M. L., &amp; Jennions, M. D. (2019). <a href=\"https:\/\/journals.plos.org\/plosbiology\/article?id=10.1371\/journal.pbio.3000127\">Evidence that nonsignificant results are sometimes preferred: Reverse P-hacking or selective reporting?<\/a>. <i>PLoS biology<\/i>, <i>17<\/i>(1), e3000127.<\/div>\n<\/li>\n<li>\n<div class=\"gs_citr\" tabindex=\"0\">Snyder, C., &amp; Zhuo, R. (2018). <a href=\"https:\/\/www.nber.org\/papers\/w25058\"><i>Sniff Tests in Economics: Aggregate Distribution of Their Probability Values and Implications for Publication Bias<\/i><\/a> (No. w25058). National Bureau of Economic Research.<\/div>\n<\/li>\n<\/ul>\n<p>These papers concern test of concern in studies, not tests for main findings that are undesired.<\/p>\n<p>I first heard of this concept in ~2012 from Neuroskeptic:<\/p>\n<div class=\"oceanwp-oembed-wrap clr\">\n<blockquote class=\"twitter-tweet\" data-width=\"550\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">New post: <a href=\"http:\/\/t.co\/1k7pCpkJ\">http:\/\/t.co\/1k7pCpkJ<\/a> The dangers of reverse publication bias &#8211; when everyone wants to publish negative results<\/p>\n<p>&mdash; Neuroskeptic \ud83c\uddfa\ud83c\udde6 (@Neuro_Skeptic) <a href=\"https:\/\/twitter.com\/Neuro_Skeptic\/status\/172584878645317632?ref_src=twsrc%5Etfw\">February 23, 2012<\/a><\/p><\/blockquote>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/div>\n<p>Some have also tried to explain away replication failures as being a kind of reverse p-hacking:<\/p>\n<div class=\"oceanwp-oembed-wrap clr\">\n<blockquote class=\"twitter-tweet\" data-width=\"550\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">There is indeed evidence of reverse publication bias <a href=\"https:\/\/t.co\/SvkRmar5Rg\">https:\/\/t.co\/SvkRmar5Rg<\/a><\/p>\n<p>&mdash; Dan Quintana (@dsquintana) <a href=\"https:\/\/twitter.com\/dsquintana\/status\/1126815404217786369?ref_src=twsrc%5Etfw\">May 10, 2019<\/a><\/p><\/blockquote>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/div>\n<div class=\"oceanwp-oembed-wrap clr\">\n<blockquote class=\"twitter-tweet\" data-width=\"550\" data-dnt=\"true\">\n<p lang=\"en\" dir=\"ltr\">Is there a &quot;reverse publication bias&quot; in favour of the efficient market hypothesis?<a href=\"https:\/\/t.co\/GvG6miDsY9\">https:\/\/t.co\/GvG6miDsY9<\/a> <a href=\"https:\/\/t.co\/4dvUpBNDc7\">pic.twitter.com\/4dvUpBNDc7<\/a><\/p>\n<p>&mdash; Lionel Page (@page_eco) <a href=\"https:\/\/twitter.com\/page_eco\/status\/956048519357456385?ref_src=twsrc%5Etfw\">January 24, 2018<\/a><\/p><\/blockquote>\n<p><script async src=\"https:\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/div>\n","protected":false},"excerpt":{"rendered":"<p>See also Sesardi\u0107&#8217;s conjecture, a related idea. Publication bias as normally considered is really positive publication bias, i.e., the bias is away from zero, towards finding larger than reality results. There is another form, however, more rare, called reverse publication bias, or negative publication bias, where published results are biased towards the null. This pattern [&hellip;]<\/p>\n","protected":false},"author":17,"featured_media":8633,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2591,2049],"tags":[2829],"class_list":["post-8629","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-intelligence-iq-cognitive-ability","category-metascience","tag-reverse-publication-bias","entry","has-media"],"_links":{"self":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/8629","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/comments?post=8629"}],"version-history":[{"count":4,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/8629\/revisions"}],"predecessor-version":[{"id":8634,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/8629\/revisions\/8634"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/media\/8633"}],"wp:attachment":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/media?parent=8629"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/categories?post=8629"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/tags?post=8629"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}