Categories
Metascience

Media darling scientists

Being on Twitter you quickly run into some people calling themselves scientists, but who seem to not be doing much science at all and instead spending their time on politics and self-promotion, not to mention book sales. Some years ago, some economists did some numbers on this and came up with the Kardashian Index (original paper, commentary here, here, online calculator). This is essentially a rescaled residual score of how many more followers on Twitter a scientist has compared to the expected number based on his citations as measured by Google Scholar. One can get the idea from a single plot from this post:

The person displayed being JFG, who used to be a neuroscientist postdoc, but who dropped out. He has ~12 research publications and over 10k Twitter followers (~35k as of writing), mainly because he has a popular alt-right talk show (The Public Space, ~23k subs).

The most extreme cases of Kardashians are people who put “Dr” or “PhD” prominently on their Twitter accounts, and who always talk about what science supposedly says, but when one searches for their publications, one finds not much at all. Adam Rutherford might serve as an example here, though they don’t have to be left-wing.

Like Action Amy of NYT, he follows me, presumably because I supply good content (read: examples of evil pseudoscience) for his upcoming books and paid lectures. Nonetheless, if we search for his body of scholarly works, we search mostly in vain. There is no Google Scholar page (search results), and there is only an autogenerated ResearchGate profile, which has mostly non-research work. Adam used to work for a journal as a journalist of sorts, so he wrote a bunch of articles in a journalistic role, which are being indexed by RG. Nonetheless, he does have a few published papers one can find with some work (e.g. 2001 middle author, 2004 first author, 2016 middle author). If someone wants more on Adam, a very unflattering description is given here.

It would be nice to have a larger study of scientists and their various social networking/media accounts. It would make for a very interesting network analysis. One could probably assign a politics score to each scientist based on their network position and behavior (retweets, citations etc.). There have been some attempts at this kind of thing before, but they weren’t entirely good. I am certain I found one once that displayed e.g. Richard Lynn, but I can’t seem to find it again. Contact me if you know some.

Categories
Economics History Metascience Politics Sociology

Political bias in science: quotes from Gunnar Myrdal’s 1944 book

Gunnar Myrdal, who was Swedish, expresses an early version of the views the media repeat endlessly these days. To give some examples:

White prejudice and discrimination keep the Negro low in standards of living, health, education, manners and morals. This, in its turn, gives support to white prejudice. White prejudice and Negro standards thus mutually ‘cause’ each other.

The treatment of the Negro is America’s greatest and most conspicuous scandal. It is tremendously publicized, and democratic America will continue to publicize it itself. For the colored peoples all over the world, whose rising influence is axiomatic, this scandal is salt in their wounds.

In this sense the Negro problem is not only America’s greatest failure but also America’s incomparably great opportunity for the future. If America should follow its own deepest convictions, its well-being at home would be increased directly. At the same time America’s prestige and power abroad would rise immensely.

The study of women’s intelligence and personality has had broadly the same history as the one we record for Negroes. As in the case of the Negro, women themselves have often been brought to believe in their inferiority of endowment. [all quotes from The American Dilemma]

Less known are his statements on the influence of political bias about experts in academia. These are very prescient given recent years’ debates. Some other quotes from his massive 1944 book (An American Dilemma: The Negro Problem and Modern Democracy):

In general, poor people are not radical and not even liberal, though to have such political opinions would often be in their interest. Liberalism is not characteristic of Negroes either, except, of course, that they take a radical position in the Negro problem. We must guard against a superficial bias (probably of Marxian origin) which makes us believe that the lower classes are naturally prepared to take a broad point of view and a friendly attitude toward all disadvantaged groups. A liberal outlook is much more likely to emerge among people in a somewhat secure social and economic situation and with a background of education. The problem for political liberalism-if, for example, we might be allowed to pose the problem in the practical, instead of the theoretical mode-appears to be first to lift the masses to security and education and then to work to make them liberal.

The paramount practical importance of scientific research on the Negro is apparent for improvement of interracial relations. It is no accident that popular beliefs are biased heavily in a direction unfavorable to the Negro people-because they are steered by white people’s need for justification of the caste order. And it is, consequently, no accident either that scientific research, as it is progressing, is unmasking and rejecting these beliefs and giving rational reasons for beliefs more favorable to the Negroes. It is principally through encouraging research and through exposing the masses of people to its results that society can correct the false popular beliefs- by objectivizing the material out of which beliefs are fabricated. Seen in long-range perspective, a cautious optimism as to the results of gathering and spreading true information among the American people in racial terms seems warranted. The impression of the author is that the younger, and better educated, generation has, on the whole, somewhat fewer superstitious beliefs, and that, during the last decade at least, the racial beliefs have begun to be slowly rectified in the whole nation.

It should by this time be clear that it is the popular beliefs, and they only, which enter directly into the causal mechanism of interracial relations. The scientific facts of race and racial characteristics of the Negro people are only of secondary and indirect importance for the social problem under ·study in this volume. In themselves they are only virtual but not actual social facts. “.. to understand race conflict we need fundamentally to understand conflict and not race.” We have concluded, further, from the actual power situation in America that the beliefs held by white people rather than those held by Negroes are of primary importance.

On the nature of political bias in science [Appendix 2: 1 A Methodological Note on Facts and Valuations in Social Science]:

The underlying psychology of bias in science is simple. Every individual student is himself more or less entangled, both as a private person and as a responsible citizen, in the web of conflicting valuations, which we discussed in Appendix I. Like the layman, though probably to a lesser extent, the scientist becomes influenced by the need for rationalizations. The same is true of every executive responsible for other people,s research and of the popular and scientific public before which the scholar performs, and whose reactions he must respect. Against the most honest determination to be open-minded on the part of all concerned and, primarily, on the part the scientists themselves, the need for rationalization will tend to influence the objects chosen or research, the selection of relevant data, the recording of observations, the theoretical and practical inferences drawn and the manner of presentation of results.

The method of detecting bias also is simple. As the unstated premises are kept hidden, the inferences drawn from them and from the factual data contain logical flaws. The general method of detecting biases is, therefore, to confront conclusions with premises and find the non sequitur which must be present if inferences are biased. If all premises arc not stated explicitly, the inferences must be inconclusive. Thia method works as long as the biases are restricted to the plane of inferences. If the basics have influenced the very observations, so that the observed data are wrongly perceived and recorded, the method is to repeat the observations. If they have influenced the selection of data collected, the viewpoints and hypotheses applied, or the demarcation of the field of study, the check consists in the application of alternative hypotheses and the widening of the scope of research to embrace the neglected fields. The awareness of the problem of bias is a most important general protection.

and

In the course of a general movement in the American social sciences toward increasing emphasis upon the “environment” as a cause of differences between social groups the scientific treatment of the Negro problem has, during the last few decades, become vastly more friendly to the Negroes. Without any doubt many white scientists in the field, perhaps the majority, have attached their research interests to the Negro problem or to various aspects of it because of a primary reform interest. In the national ethos there is traditionally, as we often have occasion to point out, a strong demand for “fair play” and for consideration toward “the underdog.” Since Negroes are severely suppressed, even today, and since by virtue of that fact they often fall below the mark in conduct and accomplishments, and since public opinion is still prejudiced against the Negroes, even a friendliness which stands out an exceptional may allow views which are rather on the unfriendly side of true objectivity. The range of scientific opinions, therefore, docs not even today necessarily include the unbiased opinion.

and specifically:

(c) The Scale of Radicalism-Conservatism. The place of the individual scientist in the scale of radicalism-conservatism have always had, and still has, strong influences upon both the selection of research problems and the conclusions drawn from research. In a sense it is the master scale of biases in the social sciences. It can be broken up into several scales, mutually closely integrated: equalitarianism–aristocratism, environmentalism–biological determinism, reformism–laissez-faire, and so forth. There is a high degree of correlation between a person’s degree of liberalism in different social problems. Usually the more radical a scientist is in his political views, the more friendly to the Negro cause he will feel and, consequently, the more inclined he will be to undertake and carry out studies which favor the Negro cause. The radical will be likely to take an interest in refuting the doctrine of Negro racial inferiority and to demonstrate the disadvantages and injustices inflicted upon the Negro people.

The tendency toward increased friendliness to the Negro people, already referred to, is undoubtedly related to a general tendency during the last few decades, in American society and its social science, toward greater liberalism. In a particular problem where public opinion in the dominant white group is traditionally as heavily prejudiced in the conservative direction as in the Negro problem, even a radical tendency might fail to reach an unprejudiced judgment; whereas under other circumstances or in other problems the objective truth might lie beyond the most extreme conservative position actually held. The prevalent opinion that a “middle-of-the-road” attitude always gives the best assurance of objectivity is, thus, entirely unfounded.

Categories
Genomics Metascience

Admixture analysis and genetic causation: some quotes from the literature

A common comment on bias in scientific peer review is that reviewers don’t usually say openly they are applying double standards. Instead, they just silently increase their standards. If their bias against some finding is strong, the evidential burden to meet goes to infinity, making sure that nothing is rigorous enough to pass review. A case in point of this behavior was very clear in our attempts to get an admixture analysis for race and intelligence published. Although admixture analysis is commonplace in medical genetics and in scientific anthropology, somehow the interpretation of such findings is totally different when one changes the trait.

For instance, one really hostile reviewer recently wrote:

2. Second, I made the point that the fundamental logic of the study is weak. The authors simply state that they are following accepted protocol in genetic epidemiology. For one thing, the authors provide no basis to believe that their study follows accepted protocol in genetic epidemiology – their approach is certainly not widely accepted as a means of demonstrating a causal influence of continental ancestry on cognitive/behavioural traits; for another, saying ‘this is the way things are done’ does not rebut my point that the logic of the study is flawed.

[…]

4. but I do think that any such enquiries have to be held to a very high standard of evidence, given the potential social harms of misguided findings. The evidence presented here is not of a high standard at all.

I have discussed the topic of causality and admixture results at length (e.g. in my long PING write-up, and in other places), and it’s also done in this version of the paper (reviewer never commented on that, as to be expected). However, we can easily disprove his claim that admixture findings are not generally taken to indicate causality. We thank this particular reviewer for openly admitting his double standards.

The collection of quotes below is obviously not exhaustive. Indeed, I compiled most of these in about two hours. One can find 100s of such quotes if determined to spend a day or two. To find such quotes, one can use search queries like this one for African Americans.

Medical genetics

African Americans and health outcomes

In the Atherosclerosis Risk in Communities (ARIC) Study, African Americans are twice as likely as whites to develop incident type 2 diabetes—a disparity which persists even after extensive adjustment for socioeconomic status (SES) and behavioral risk factors [4]. This persistent disparity suggests that genetic factors may contribute to ethnic differences in susceptibility to type 2 diabetes.

[…]

Given the observed ethnic/racial disparities in diabetes prevalence, we hypothesized that some diabetes susceptibility alleles are present at higher frequency in African Americans than in European Americans, resulting in association between genetic ancestry and diabetes risk that is independent of its association with other non-genetic risk factors for type 2 diabetes. Thus we sought 1) to establish the association of genetic ancestry with diabetes and related quantitative traits in African Americans, after accounting for the non-genetic risk factors, and 2) to identify diabetes susceptibility loci by conducting a genome-wide admixture mapping scan.

[…]

In summary, in community-based populations with more than 7,000 African Americans, we found that genetic ancestry is significant associated with type 2 diabetes above and beyond the effects of markers of SES, and we detected several suggestive loci that may harbor genetic variants modulating diabetes risk. These results suggest that in African Americans, genetic ancestry has a significant effect on the risk of type 2 diabetes that are independent of the contribution of SES, but that no single locus with a major effect explains a large portion of the observed disparity in diabetes risk between African Americans and European Americans. In addition, they suggest that genetic measured African ancestry contributes to the risk of type 2 diabetes via both genetic and non-genetic pathways. The effect of ancestry on any individual locus in the genome is likely to be modest, but in aggregate, differences in ancestry may contribute substantially to the observed ethnic disparity in risk of type 2 diabetes.

This study is particularly noteworthy in that the authors explicitly present SIRE gaps that remain after extensive (sociologist fallacy style) controls as being evidence of genetic causation. They afterwards then hypothesize an association between genetically measured ancestry and outcome risk, which they then find. This is basically the same reasoning used by Jensen in in 1969 and forwards.

We have demonstrated that genetic ancestry may serve as a biomarker for identifying smokers who would benefit from targeted counseling regarding smoking cessation [41], [42]. One important implication of our findings is that there may be rare genetic variants relevant to smoking associated lung function decline that are population-specific and which co-vary with genetic ancestry [43]. While we cannot rule out that some of these associations may be in part due to environmental factors which co-vary with ancestry, these results highlight the scientific advantages of studying racially mixed populations. Future analyses should include admixture mapping to identify genomic regions associated with rate of lung function decline.

In summary, a consistent association of African ancestry with asthma risk was observed in a large case-control sample of self-reported African American subjects. Although confounding effects attributable to other relevant risk factors cannot be ruled out, we replicate previous findings and support the notion that ethnic disparities in asthma incidence are affected, in part, by genetic determinants. Frequency differences for risk alleles across populations and/or differential gene-environmental interactions may lead to differential disease susceptibility.

Due to this heterogeneity, genetic admixture analysis offers a unique opportunity for studying the role of genetic factors within a single, admixed population, independent of social factors, and comorbidities. Ancestry informative markers, AIMs, are genetic loci showing alleles with large frequency differences between populations that can be used to estimate bio-geographical ancestry at the level of the population and individual. Ancestry estimates at both the subgroup and individual level can be directly instructive regarding the genetics of the phenotypes that differ qualitatively or in frequency between populations (Shriver et al., 2003). Specifically, an association between genetic ancestry and a disease phenotype within an admixed group such as AAs may be an indicator of genetic factors underlying differential expression among racial groups (Peralta et al., 2010).

A greater proportion of African genetic ancestry is independently associated with higher FG
levels in a non-diabetic community-based cohort, even accounting for other ancestry
proportions, obesity and SES. The results suggest that differences between African-Americans
and whites in type 2 diabetes risk may include genetically mediated differences in glucose
homeostasis.

The mechanisms that underlie differences in sleep characteristics between European Americans (EA) and African Americans (AA) are not fully known. Although social and psychological processes that differ by race are possible mediators, the substantial heritability of sleep characteristics also suggests genetic underpinnings of race differences. We hypothesized that racial differences in sleep phenotypes would show an association with objectively measured individual genetic ancestry in AAs.

[…]

Ancestry-phenotype association tests, which quantify associations between measured genetic ancestry and a phenotype in an admixed population, like AAs, can be used to test the extent to which the genetic characteristics underlying race may be responsible for observed population level differences.39,40 In the context of sleep, ancestry-phenotype association tests assume that multiple genetic variants, each with small effects on sleep, may have different allele frequencies in different continental populations that contributed to the admixed population. Because individuals from the admixed population inherit varying proportions of their genome from different ancestral populations, one expects contribution from any one ancestral population to show a wide range of variation (theoretically, spread between 0–100%). Any association between ancestry and a phenotype in an admixed group, then, indicates that multiple variants across the genome that have been inherited from one particular ancestral population are related to variation in the phenotype.3945 In this manner, objectively measured genetic ancestry enables us to test the uniquely genetic facet of “race” parsed from the cultural, behavioral, and psychosocial aspects that may be responsible for the observed phenotypic differences.

[…]

By utilizing the genetic variability attributable to continental admixture, we show for the first time that visually scored percent SWS and NREM EEG delta are associated with %AF in AAs. Even after adjusting for several demographic, socioeconomic and clinical covariates, %AF explained between 9% and 11% of the variance in SWS in AAs. These results show that AAs have inherited multiple alleles (either few alleles of large effect sizes or several alleles of moderate to low effect sizes) from their African ancestors that may pre-dispose them to lower percent SWS. This association between measured genetic ancestry and SWS clearly establishes a partial genetic basis underlying the observed racial differences in this dimension of sleep.

The authors even write out the Jensen logic in the abstract.

The role of genetic predisposition in this disparity is supported by two admixture mapping

studies of AAs which demonstrated that greater proportion of European ancestry was inversely
associated with fibroids in AA women.

Latin Americans and health

Some recent studies, which also used AIMs, demonstrated/ suggested that the genomic Amerindian ancestry may be protective against hypertension in women from the United States [10] , protective against metabolic syndrome in the population of Costa Rica [11] and protective against Alzheimer’s disease in Brazilian population [12] . Furthermore, recent studies in the Brazilian population showed that Amerindian individuals had lesser arterial stiffness and hypertension [13 – 14] . These studies suggest that lower risk of diseases studied in individuals with Amerindian ancestry may be due to the existence of protective genetic factors associated with this ancestry.

Significant questions remain unanswered regarding t he genetic versus environmental contributions to racial/ethnic differences in sleep and circadian rhythms. We addressed this question by investigating the association betw een diurnal preference, using the MorningnessAEveningness questionnaire (MEQ), and ge netic ancestry within the Baependi Heart Study cohort, a highly admixed Brazi lian population based in a rural town. Analysis was performed using measures of ance stry, using the Admixture program, and MEQ from 1,453 individuals. We found a n association between the degree of Amerindian (but not European of African) ancestry and morningness, equating to 0.16 units for each additional percent of Amerindian ancestry, after adjustment for age, sex, education, and residential zone. To our knowledge, this is the first published report identifying an association b etween genetic ancestry and MEQ, and above all, the first one based on ancestral contrib utions within individuals living in the same community. This previously unknown ancestral d imension of diurnal preference suggests a stratification between racial/ethnic gro ups in an as yet unknown number of genetic polymorphisms.
The authors essentially cover the entire reasoning in their abstract.

Anthropology

Pygmy height

Note the title “genetic determination”!

Considering a subset of 213 individuals for which DNA was available, we were able to formally compare the individual variation in height with the neutral genetic variation among individuals from the different Pygmy and Non-Pygmy populations.

Controlling for the binary categorization of individuals as Pygmies or Non-Pygmies, as well as for population substructure, we found strongly significant positive correlations between Pygmy individuals’ stature and their levels of admixture with the Non-Pygmy gene-pool estimated using the clustering software STRUCTURE. This result suggests that the major difference in average stature observed between Central African Pygmy and Non- Pygmy populations is likely determined by complex genetic factors.

In this context, Genome Wide Association studies and Admixture Mapping methods will likely reveal the genetic loci involved in the determination of the differences of average height found in existing African Pygmy and Non-Pygmy populations. This will further help us to better understand the determination and evolution of height variation among human populations.

We observed extensive and significant genetic and phenotypic differentiation (Figure 1, Figure 2, Figure S1) and varying levels of admixture among the Pygmy and Bantu populations. Average levels of Bantu ancestry, as determined by STRUCTURE (K = 2), in the three Western Pygmy populations were 27% (Bakola), 35% (Baka), and 49% (Bedzan) with individual values ranging from 16–73%. Average levels of Pygmy ancestry in the three Bantu populations were <1% (Lemande), 2% (Tikar), and 7% (Ngumba), with individual values ranging from 0–39%. We also observed a highly significant correlation between ancestry and height (p = 5.047×10−18) after correcting for the effect of sex (full model r2 = 0.7411, r2 for sex = 0.4247; r2 for ancestry = 0.3164). In addition, the effect of ancestry remains significant in a model that also includes Pygmy-Bantu ethnicity as a covariate (p = 3.8×10−5). These results are consistent with Becker et al. [21] and indicate a strong genetic influence on height. Similar findings were also observed using Pygmy samples only (pancestry = 0.000216; full model r2 = 0.5066; r2 sex = 0.3744; r2 ancestry = 0.1322) and the independent set of genome-wide microsatellite markers described in Tishkoff et al. [9] (data not shown).

We used the results from ADMIXTURE to estimate individual ancestry proportion ( K = 5 for esti- mating pygmy ancestry, and K = 8 for Asian ancestry) and its correlation with adult height for 43 men and 27 women from the different pygmy groups of the Philippines (Aeta, Agta, and Batak) and for the nonpygmy groups (Tagbanua, Zambales, Casiguran). Because K = 5 separates negritos and Asians, we used individual “negrito” ancestry proportion to correlate with their adult height. This procedure allows us to estimate the effect of genetic contribution on adult height.

As expected, mean stature estimates for the Batwa (66 males, 152.9 cm; 103 females, 145.7 cm) were lower than those for the Bakiga (20 males, 165.4 cm; 41 females, 155.1 cm; Fig. 2 B ). Batwa stature is significantly positively correlated with the proportion of Bakiga admixture: for males, females, and for all samples combined after regressing out the sex effect (Fig. 2 C – E ), confirming a genetic basis for the African pygmy phenotype (6, 12).

[…]

We can draw four primary conclusions from our analyses. ( i ) The African pygmy phenotype has a genetic basis, rather than a solely environmental one, based on the positive correlation between stature and Bakiga admixture for Batwa individuals raised in Batwa communities (Fig. 2 C – E ). These results confirm those obtained from other African rainforest hunter-gatherer populations by Becker et al. (12) and Jarvis et al. (6) and are consistent with individual case observations from Cavalli-Sforza (4).

Although environmental variation is an important factor influencing adult height, such influences are considered insufficient to account fully for observed population differences. Some African populations are considerably taller than others, for example, despite experiencing poorer nutrition and elevated levels of pathogen exposure (Deaton, 2007), suggesting that such differences may have a genetic basis. To date, very few studies have addressed this issue. Notable exceptions are studies investigating the difference in height observed between the Baka pygmies of Cameroon and taller neighbouring non-Pygmy populations (Becker et al., 2011; Jarvis et al., 2012). Both of these studies showed that Pygmy individuals who were genetically more similar to non-Pygmy individuals (i.e. higher levels of genetic admixture) were taller. Most recently, Perry et al. (2014) have shown that the pygmy phenotype likely arose several times independently due to positive natural selection for short stature. Additional evidence for genetic factors underlying population differences in height come from a Korean population (Cho et al., 2009).[…]

Substantial levels of non-Pygmy genetic admixture have been observed across Central African Pygmy populations [ 24 – 2 7 , 29 – 31 ] , correlating positively with adult standing height [ 32 – 34 ] . The general genetic difference be tween Pygmies and non-Pygmies together with the correlation of genetic admixture and standing height suggests that adult body size differences among Central African Pygmies and neighboring non-Pygmies are attributable in large part to genetic factors, arguing against a view that diminutive Central African pygmy body size is the consequence solely of phenotypic plasticity in a challenging nutritional and parasitic environment [ 8 ].

[…]

Our findings accord with prior observations [ 59 , 63 ] that while Pygmy body size is generally proportionally reduced relative to non-Pygmies, their leg lengths are significantly shorter relative to their trunk length. Importantly, our results provide further support for an appreciable genetic component to the determination of body size differences between Pygmies and non-Pygmies, as implied by the correlations observed between the different measures and inferred levels of non-Pygmy admixture that replicate those reported previously for adult standing height [ 32 – 34 ] .

Note that the authors changed their wordings a bit in the published version. Maybe they too encountered some funny reviewers!

Amerindian-descent physical appearance

A number of studies look at admixture in Amerindian populations, relating both macro-race/continental ancestry to phenotypes as well as sub-Amerindian clusters.

Furthermore, studies of regional human genome diversity, and its bearing on phenotypic variation, have so far been strongly biased towards European-derived populations17. The study of populations with non-European ancestry is essential if we are to obtain a more complete picture of human diversity. Latin America represents an advantageous setting in which to examine regional genetic variation and its bearing on human phenotypic diversity18, considering that the extensive admixture resulted in a marked genetic and phenotypic heterogeneity2,3,19. Relative to disease phenotypes, the genetics of physical appearance can be viewed as a model setting with distinct advantages for analyzing patterns of genetic and phenotypic variation. Many physical features are relatively simple to evaluate, show substantial geographic diversity and are highly heritable. We have previously shown that variation at a range of physical features correlates with continental ancestry in Latin Americans19 and have identified genetic variants with specific effects for a number of features20,21,22.

[…]

We infer the timings of these genetic contributions and relate them to historically-attested migrations, for example providing compelling new evidence of widespread ancestry from undocumented migrants during the colonial era. We further show how differences in Native and European sub-continental ancestry components are associated with variation in physical appearance traits in Latin Americans, highlighting the impact of regional genetic variation on human phenotypic diversity.

Categories
Metascience Peer review

Peer review and innovation

This little read paper from 2002 is worth quoting at length. It underlines the inability of peer review to identify important studies, and its role in guarding the status quo in the field. Based on such thinking, some people have come to the conclusion that one should simply do away with peer review and return to the normal scientific mode of publishing, where is the editor review (i.e. the editor reads it over and if it seems fine, that’s it).

It is doubtful, however, that the difficulty of publishing a paper claiming some innovative idea, concept or model would be sufficient to prevent eventual publication unless the paper was of such a poor quality and contained obvious inconsistencies that virtually all referees would move for rejection. Persistence and perseverance and sending the often-rejected paper to a sufficiency of journals normally results in publication: even though such a process may take several years. Therefore, it would be difficult to claim that the idiosyncrasies of the academic publication process would inhibit innovative thinking or writing: such activities would take place in any case; it is their publication which is delayed. One consequence would be that the innovative article is most likely to appear in a second or third ranked journal. As such publications are less widely read (especially by newspaper reporters and commentators, who review the first rank journals as a matter of course) the dissemination of the novel idea is hindered and often completely buried. In this sense the peer review process has prevented the widespread broadcasting of the new idea with the result that its manifestation and implications may be lost for many years. However, with modern retrieval techniques of literature searching (using the internet, search facilities and ‘hot links’) and the increasing number of meta-analyses which are conducted, it is not unlikely that, providing the essence of the innovative idea is implicit in the title and abstract, the article would be ‘fished out’ from the sea of dross surrounding it, resulting in an elevation to its rightful place in the ‘sun’.

It could be held that the pressure on academics to publish papers that are cited (another form of peer review) is often a determining influence on what is published. Papers which contain a method that becomes adopted by the field (an innovative departure) are a prime target for an ambitious academic. But papers that have innovative theoretical ideas are also well cited as such ideas are subjected to various tests and experimentation. There is, however, little correlation between what reviewers think about the quality of a paper (by a prediction of its citability) and the actual citations obtained, 1 which means that the targeting of a paper for its future citability is a dubious means of achieving such acclaim. So, as the performance of an academic is measured by successful and cited publications, this criterion drives the research process towards a ‘me-tooism’ approach, which is safe and likely to be cited by others working to a similar driving dogma in a similar or related subject area. This results in the small step change type of innovation which may, by happenstance, lead to a completely new paradigm, but which generally serves merely to expand the literature of the field.

Peer review of publications does not welcome, support or promote innovation but neither does it prevent it. Such novelty as does occur relies on the foresight and determination of the author. People in general are resistant to change and the introduction of that which is deemed foreign. As much innovation is strange at first sight, resistance to its promulgation may be considered natural. Innovative work survives because of its intrinsic merit: it succeeds as people become familiar with its advantages and prospects. It also emerges when the necessity to achieve a new goal has been clearly enunciated with accompanying funding.

Categories
Metascience

Do Republic professors hide their voting intentions?

There’s a lot of evidence for left-wing dominance of academia. One study (Langbert 2018) looked at registered voters and finds quite unbelievable differences:

In this article I offer new evidence about something readers of Academic Questions already know: The political registration of full-time, Ph.D.-holding professors in top-tier liberal arts colleges is overwhelmingly Democratic. Indeed, faculty political affiliations at 39 percent of the colleges in my sample are Republican free—having zero Republicans. The political registration in most of the remaining 61 percent, with a few important exceptions, is slightly more than zero percent but nevertheless absurdly skewed against Republican affiliation and in favor of Democratic affiliation. Thus, 78.2 percent of the academic departments in my sample have either zero Republicans, or so few as to make no difference.

My sample of 8,688 tenure track, Ph.D.–holding professors from fifty-one of the sixty-six top ranked liberal arts colleges in the U.S. News 2017 report consists of 5,197, or 59.8 percent, who are registered either Republican or Democrat. The mean Democratic-to-Republican ratio (D:R) across the sample is 10.4:1, but because of an anomaly in the definition of what constitutes a liberal arts college in the U.S. News survey, I include two military colleges, West Point and Annapolis.1 If these are excluded, the D:R ratio is a whopping 12.7:1.

My guess here is that republican professors are likely to avoid registering to vote if this fact can be discovered by their colleagues, who perhaps suspect something and are likely to want to retaliate. However, this bias should not be present in anonymous surveys of professors, so can we find a similar ranking based on that? Yep, Klein and Stern (2005) reported some results:

In Spring 2003, a large-scale survey of American academics was conducted using academic association membership lists from six fields: Anthropology, Economics, History, Philosophy (political and legal), Political Science, and Sociology. This paper focuses on one question: To which political party have the candidates you’ve voted for in the past ten years mostly belonged? The question was answered by 96.4 percent of academic respondents. The results show that the faculty is heavily skewed towards voting Democratic. The most lopsided fields surveyed are Anthropology with a D to R ratio of 30.2 to 1, and Sociology with 28.0 to 1. The least lopsided is Economics with 3.0 to 1. After Economics, the least lopsided is Political Science with 6.7 to 1. The average of the six ratios by field is about 15 to 1. Our analysis and related research suggest that for the the social sciences and humanities overall, a “one-big-pool” ratio of 7 to 1 is a safe lower-bound estimate, and 8 to 1 or 9 to 1 are reasonable point estimate. Thus, the social sciences and humanities are dominated by Democrats. There is little ideological diversity. We discuss Stephen Balch’s “property rights” proposal to help remedy the situation.

The list of overlapping fields is smallish, but sufficient for our purposes. It looks like this:

Field Survey Registered Ratio
anthropology 30.2 56 1.85
economics 3 5.5 1.83
history 9.5 17.4 1.83
philosophy 13.5 17.5 1.30
political science 6.7 8.2 1.22
sociology 28 43.8 1.56

 

If we look at the relative agreement, it is convincingly large at r = .98. However, we see that the registration method consistently produces larger ratios, suggesting some hiding of republicans. The average ratio is 1.60.

Categories
Metascience

Wanted: scientific immune system to identify weak studies getting lots of attention

In the interest of keeping the scientific enterprise towards finding truth, it is important to reduce the impact of problematic studies in the scientific literature. Studies can be problematic in many ways (e.g. lacks a control for genetic confounding in social science), but one relatively simple problem to automatically identify is low precision due to small sample size. Since such studies are too imprecise to really tell us much about reality, they should be given quite little attention. Unfortunately, something like the opposite might be true, with small studies with flashy results being given worldwide media attention.

Here’s my idea. We need to have a preferably public database of all scientific papers with fulltext as they are published. It doesn’t have to have complete backlog coverage because we are just trying to reduce incoming damage to the literature from uninformative papers, it is too late for the old ones. As each new paper comes out, it is put in the database along with extracted metadata from it such as sample sizes, statistical tests, standard errors, and whatever information one can find. Then we calculate some kind of overall paper informativeness score, which could be simply something like the replication index or median observed power. We also monitor every papers’ altmetrics score, which tracks attention to papers. Then we identify papers with weak statistics and high attention. The scientific team seeing the results can then write a specific response to that particular paper in an attempt to reduce its impact on the literature.

I’ll give two case studies of what I have in mind.

Yet another trans-generational epigenetics study

So what’s the study? Besides being a trans-generational epigenetics study, itself a red flag, the study does not list the sample size anywhere, not even in the methods section. However, we can find it by looking at the reported degrees of freedom, which are at 10. My guess is that it is a balanced 2×6 study, i.e. a study of 12 mice is causing worldwide media attention! Looking at the stats makes us even more worried. There are 10 p values reported exactly, these are: p = 0.17, p = 0.10, p = 0.07, p = 0.018, p = 0.04, p = 0.01, p = 0.01, p = 0.28, p = 0.19, p = 0.08. All values between 0.01 and 0.28, very suspicious! We can’t do a formal test for too little variation here because some of these test related hypothesis, thus the values are correlated by design, violating the assumptions of the TIVA test (independence). But we are still pretty skeptical because a study of n = 12 can produce pretty much any result imaginable and is basically useless.

Trusting co-partisans more

Study is being given attention by the heterodox community on Twitter (complete listing of people posting the link on Twitter):

The methods section:

2.1. Participants
American residents over 18 years of age who speak English were recruited on Amazon Mechanical Turk. All participants provided demographic information (see supplementary materials). 154 participants completed the first part of the task (Learning Stage), out of which 97 participants (34 females and 63 males, aged 20-58 years M = 34.81, SD = 9.59) completed the second stage (Choice Stage). All participants were paid $2.50 for completing the first stage of the experiment and were told they could earn a bonus of $2.50 to $7.50 based on their performance. Thus, they had an incentive to perform well. Because in reality participant performance was held constant at 50% all participants who completed the entire experiment were paid a $5 bonus.

I.e. a design they could have easily collected more data for. Why stop at such a low value? Very suspicious of optional stopping. The splashy result is is described as “t(96) = -2.10, p = .038, d = -.37″. I suggest there is no reason to read the rest of the paper until they bump the sample size.

Categories
Math/Statistics Metascience

Making better use of the scientific literature: large-scale automatic retrieval of data from published figures

Science is a set of related methods that aim at finding true patterns about the world. These methods are generally designed so as to remove noise from random circumstances (the traditional focus of statistics) and human biases. Current practices are currently not very good at the second part due to the innumerable ways human biases result in biased findings (see e.g. discussion for social psychology in Crawford and Jussim’s new book). However, I feel confident that many of these biases can be strongly reduced with the advent of several new tools and practices: 1) by the development of meta-analytic tools to properly summarize existing possibly biased research (e.g. p-curve, z-curve, pet-peese, TIVA, R index), 2) registered replication reports that remove outcome bias in peer review, 3) the increasing reliance on automated tools for checking the validity of scientific works (statcheck, GRIM, SPRITE etc.), 4) increasing awareness of the presence of very substantial ideological biases in the scientific community.

Here I want to suggest a new approach that attacks a different but related angle: lack of open research data. While there is a growing movement to publish all research data, still a large chunk of data remains unpublished. Most of this data will be lost as no backups of it exist and the authors eventually die or throw away their laptops etc. However, papers endure and papers have various visualizations of the data. Some of these are scatterplots (example on the right), which allow for automatic and complete retrieval of the underlying data (unless there’s overplotting).

 

Others are visualizations of mostly summary statistics which allow for some form of data retrieval. For instance, boxplots (on left) allow for retrieval of the 25th, 50th and 75th centiles and also of any outlying datapoints (and usually the 1.5 IQRs). This kind of information can be used in meta-analyses and can also be combined with GRIM/SPRITE-type methods to verify the integrity of the underlying data.

Various other kinds of visualizations allow for all kinds of intermediate levels of data retrieval. For instance, scatterplots that vary the size of the dots by a third variable allow retrieval of at least some levels of that variable as well. Many newer PDFs contain vector graphics not raster graphics, and this allows for precise retrieval, not just approximate.

 

There already exist quite a large collection of published tools for data retrieval, including some of which are open source R or Python packages which could easily be integrated in any existing framework for data mining the scientific literature. I have found the following collections of tools:

  • https://academia.stackexchange.com/questions/7671/software-for-extracting-data-from-a-graph-without-having-to-click-on-every-singl
  • https://stats.stackexchange.com/questions/14437/software-needed-to-scrape-data-from-graph/72974
  • https://www.techatbloomberg.com/blog/scatteract-first-fully-automated-way-mining-data-scatter-plots/
Categories
Differential psychology/psychometrics Metascience

Why the race and intelligence question is still not resolved

It could probably have been resolved decades ago, and definitely within the last 10 years with genomic data, yet it is still not. Why? Essentially, it’s because of bias in academia. It begins early: data access, then there’s authors’ own publication bias, then finally editorial and reviewing bias (all caused by lack of political/belief diversity in academia). Here’s a recent example of the first, data access bias. How can there be this bias? Because academia has been hoarding the data that the public has funded them to gather, refusing to share it with anyone openly. Well, actually academics just threw away most of the data (‘we accidentally all the data’), but the data they didn’t just let perish, most of is safe-guarded for privacy reasons so others can’t use it to publish without the guy who collected it (‘steal his ideas’), and especially so nefarious characters (aka. political opponents) cannot get it.

We were applying to a large dataset that would probably be able to settle the race and intelligence, or at least, provide very strong evidence for or against genetics as cause. But instead we got:

I asked Prof. X about this project, and even though he does recognise its relevance, I am afraid that he declined to provide the data. The reason is that country C is facing a very delicate political situation at the moment, and race/ethnicity/ancestry is one of the topics at the core of debates and etc. Everyone in the country seems to be a little bit cautious when it comes to looking at ethnicity, especially regarding this such as intelligence or violence.
I am truly sorry that we will not be able to help at this opportunity, but I do wish the best of luck with your research.

And onwards we go towards applying for the next dataset.

Categories
Differential psychology/psychometrics Metascience Science

On scientific consensus

In reply to: Scott Alexander’s Learning To Love Scientific Consensus. Actually, I have planned (in my mind) a somewhat longer post on my take on the ‘correct contrarian cluster’, or how to make up your mind of what to believe on controversial topics. But I certainly don’t have time to write that now, so instead you get this post.

Regarding historical examples, this list collected by The Alternative Hypothesis is much better than the various top 10 lists you can find. Maybe somewhat with more time and knowledge of science history than me should go over it.

In so far as their examples hold up, they make a smarter argument. Instead of doing only “consensus was X, and X was wrong” arguments, which can always be attacked on the grounds that “X wasn’t really wrong”, they do arguments of “consensus was X, then consensus was Y, and now it’s X or Z”. For this argument, one does not need to agree on the truth of the matter, only the expert consensus status. If consensus changed, then no matter what the truth really is, consensus was not always in agreement with it.

A more sophisticated version of the consensus-is-right position is to argue that the consensus may not be the truth, but it follows the evidence base as a whole (slowly and wiggly). Sometimes the evidence base may point towards a falsehood for extended periods of time. Not necessarily due to any particular political or religious bias, but just due to the difficulty of some scientific questions, lack of good quality relevant data, stochastic processes among humans etc. This view is harder to attack, but also harder to positively argue for because it is really hard to show track of what the total evidence base pointed towards in, say, 1955 re., say, plate tectonics theory. Finding out requires doing a lot of reading on old material and painstakingly avoiding anachronisms. This is far beyond what most people are willing and capable of doing, and generally only a few science historians take this route (bless them!).

It is of course often hard to know what the expert consensus is. Aside from controversial topics (evolution, global warming, IQ, GMO etc., see below), scientists don’t routinely conduct large surveys of expert beliefs (they should!). So instead people rely on their general impressions of what experts believe. If they don’t read the scientific literature, what other source to rely on than the media? This gives rise a likely* media bias effect as the media tends to present experts that believe things media people believe in, and not necessarily what the experts generally believe in. So, lots of space to, say, “10,000 hours of training” researchers and lots of space to “the next early intervention will fix inequality, promise!” researchers. And then people get the impression that the consensus is something else than it really is. I wish the media would adopt public high score tables for ‘science pundits’ and give preferential treatment (attention) to those who tend to make correct predictions (an idea proposed by John Pressman here). The media and scientists should team up to make pre-registered experiments and predictions in the hostile collaboration style.

For the record, I don’t think of myself as a revolutionary scientist (or a ‘universal genius’ as a journalist recently made up). To put it in horse terms, my view is that essentially I’m betting on an winning horse early, and that this horse has a bad reputation, but that in 10 years or so, there will be widespread open acknowledgement that this is a good horse nonetheless (yes, I am talking about race and IQ). As the surveys show, experts generally do agree with me on this topic, altho I’m surely more towards the genetic side than the median. (But it does depend on who counts as an expert. Presumably most experts surveyed on this question have not studied it in depth.) Scott does not like my presentation style of these ideas because it makes it harder for nice people like Steven Pinker to work on the public image of the ideas. I understand accept this argument, but I would like to point out that when people with my views don’t speak candidly about them, the media bias effect gets stronger. Personally, I think scientific should discuss their ideas openly and frankly. In general, I think the long-term consequences of suppression of unpopular findings is a net native outcome. I think it also applies in this case, i.e. the harms of ignoring scientific findings about what works with regards to social inequality is likely to have much larger negative consequences than the alternative. Frankly, I don’t think Nazi-style totalitarian governments that target specific races/ethnicities/religions have any large chance of coming back in Western countries (as long as these are run by Europeans).

Since history repeats itself, here’s Jensen in 1969 (How much can we boost IQ and scholastic achievement?):

The question of race differences in intelligence comes up not when we deal with individuals as individuals, but when certain identifiable groups or subcultures within the society are brought into comparison with one another as groups or populations. It is only when the groups are disproportionately represented in what are commonly perceived as the most desirable and the least desirable social and occupational roles in a society that the question arises concerning average differences among groups. Since much of the current thinking behind civil rights, fair employment, and equality of educational opportunity appeals to the fact that there is a disproportionate representation of different racial groups in the various levels of the educational, occupational, and socioeconomic hierarchy, we are forced to examine all the possible reasons for this inequality among racial groups in the attainments and rewards generally valued by all groups within our society.

To what extent can such inequalities be attributed to unfairness in society’s multiple selection processes? (‘Unfair’ meaning that selection is influenced by intrinsically irrelevant criteria, such as skin color, racial or national origin, etc.) And to what extent are these inequalities attributable to really relevant selection criteria which apply equally to all individuals but at the same time select disproportionately between some racial groups because there exist, in fact, real average differences among the groups – differences in the population distributions of those characteristics which are indisputably relevant to educational and occupational performance? This is certainly one of the most important questions confronting our nation today. The answer, which can be found only through unfettered research, has enormous consequences for the welfare of all, particularly of minorities whose plight is now in the foreground of public attention. A preordained, doctrinaire stance with regard to this issue hinders the achievement of a scientific understanding of the problem. To rule out of court, so to speak, any reasonable hypotheses on purely ideological grounds is to argue that static ignorance is preferable to increasing our knowledge of reality. I strongly disagree with those who believe in searching for the truth by scientific means only under certain circumstances and eschew this course in favor of ignorance under other circumstances, or who believe that the results of inquiry on some subjects cannot be entrusted to the public but should be kept the guarded possession of a scientific elite. Such attitudes, in my opinion, represent a danger to free inquiry and, consequently, in the long run, work to the disadvantage of society’s general welfare. ‘No holds barred’ is the formula for scientific inquiry. One does not decree beforehand which phenomena cannot be studied or which questions cannot be answered.

And in 1973 (Educability and group differences):

The scientific task is to get at the facts and properly verifiable explanations. Recommendations for dealing with specific problems in educational practice, and in social action in general, are mainly a social problem. But would anyone argue that educational and social policies should ignore the actual nature of the problems with which they must deal? The real danger is ignorance, and not that further research will result eventually in one or another hypothesis becoming generally accepted by the scientific community. In the sphere of social action, any theory, true or false, can be twisted to serve bad intentions. But good intentions are impotent unless based on reality. Posing and testing alternative hypotheses are necessary stepping stones toward a knowledge of reality in the scientific sense. To liken this process to screaming ‘FIRE . . . I think’ in a crowded theatre (an analogy drawn by Scarr-Salapatek, 1971b, p. 1228) is thus quite mistaken, it seems to me. A much more subtle and complete expression of a similar attitude came to me by way of the comments of one of the several anonymous reviewers whose judgments on the draft of this book were solicited by the publishers. It summarizes so well the feelings of a good number of scientists that it deserves to be quoted at length.

* I’d like to see some more quantitative evidence as ‘general impressions’ are a low form of evidence. There was that study (n=40) of economists (Kardashian’s index) showing that those who received the most media attention has lower research output/citations, which is sorta in the right direction. Another example of this is famed scientific Neil Degrasse Tyson, who seems to have little research output. I did not check to see if there have been more of these studies, but they should be ‘fairly easy’** to conduct using Google Scholar. Maybe we should survey journalists on science topics, see which beliefs they really have, and consequently, which topics we should be attentive to bias about. See also the 2015 survey of AAAS scientists on various issues as compared with the general population.

** Nothing in science is ever as easy or fast to do as it sounds (planning fallacy). I know because I have 100 unfinished projects in my projects folder. Most of these were started on the basis of “Oh, I’ll just do this quick, easy and worthwhile study of X”. Often I do most of the data collection and analysis, but never submit it for publication. I don’t seem to learn, and recently started another 5 or so projects in a similar vein… Oh well!

Categories
Differential psychology/psychometrics Metascience

Tail effects in climate science and the pleasures of polymathy

From the interactive visualization I previously published to give foster an intuitive understanding of the concept:

Tail effects are when there are large differences between groups at the extremes (tails) of distributions. This happens when the distributions differ in either the mean or the standard deviation (or both), even when these differences are quite small. Below we see a density plot of two normal distributions with different means as well as a threshold value (vertical line). The table below the plot shows various summary statistics about the distributions with regards to the threshold. Try playing around with the numbers on the left and see how results change.

One of the pleasures of reading a very broad selection of science is that one discovers connections between fields that are not commonly connected. Sometimes these connections may give rise to important new inter-disciplinary fields or understanding, sometimes it just gives you a nice feeling of seeing the same concept in different circumstances.

Tail effects are often discussed in differential psychology because of the continued interest in group differences. These can be in whatever trait: cognitive, interest, emotional, personality-wise, and with whichever groups: social, economic, gender or racial. However, tail phenomena is more general than group differences, the two distributions can be any kind of data, including the same data from different times.

In the last year or so I have taken an increased interest in climate science. The reason is basically this, and that science denialism annoys me and incentivizes me to explore a new area of science. In fact, the whole reason I got started on science in general was that I was debating with creationists on a forum. Debating creationists effectively actually requires a fairly broad knowledge of science and philosophy. One must understand enough physics and chem. to explain how radiometric dating works, enough cosmology to explain facts related to the big bang, enough geology to explain plate tectonics, enough geology and paleontology to explain the distribution of fossils, enough evolutionary biology and genetics to explain the general ideas of evolution, and finally enough philosophy (logic, critical thinking, epistemology, philosophy of language) to spot logical errors, language and debating tricks. It isn’t exactly easy. Just listing all these areas took a number of minutes, reading the Wikipedia articles will take many hours.

Anyway, since I had been debating climate skeptics recently on a Danish-language conservative-libertarian-nationalist news aggregator, I have encountered many odd claims, which require a fairly deep knowledge of various various of climate science. E.g. to explain the facts related to Climategate, one needs an idea of temperature reconstruction with tree rings and their dating, scientific graphing, and the methods used to combine the data (principal components analysis, another method from psychometrics :) ). An issue that sometimes comes up is extreme weather. Since there was some discussion of this, including the very definition, I decided to find a review article:

The following visual explanation of extreme weather is found in the paper:

tail effects climatescience

Which showcases the general applicability of the concept. :)