- Tetlock, P. E., Kristel, O. V., Elson, S. B., Green, M. C., & Lerner, J. S. (2000). The psychology of the unthinkable: taboo trade-offs, forbidden base rates, and heretical counterfactuals. Journal of personality and social psychology, 78(5), 853.
This old article is a serious gem, and very relevant the troubles of our time. The introduction lays out the general framework well:
Research on social cognition ultimately rests on functionalist assumptions about what people are trying to accomplish when they judge events or make choices. The most influential of these assumptions have been the intuitive scientist and the intuitive economist. The former tradition depicts people whose central objective is to understand underlying patterns of causality, thereby confer-ring some advantage in anticipating life-enhancing or threatening events (cf, Kelley, 1967). The latter tradition depicts people as decision makers whose overriding goal is to select utility-maximizing options from available choice sets (Becker, 1981; Kahneman & Tversky, 1979). Although theorists often disagree sharply over how well people live up to the high professional ideals of science or economics (Mellers, Schwartz, & Cooke,1998), theorists agree in placing a normative premium on intellectual flexibility and agility. Good intuitive scientists and economists look for the most useful cues in the environment for generating accurate predictions and making satisfying decisions and quickly abandon hypotheses that do not “pan out.” Rigidity is maladaptive within both frameworks.
In this article, we explore the empirical implications of an under-explored starting point for inquiry: the notion that, in many contexts, people are striving to achieve neither epistemic nor utilitarian goals, but rather, as prominent historical sociologists have argued (Bell, 1976), are struggling to protect sacred values from secular encroachments by increasingly powerful societal trends toward market capitalism (and the attendant pressure to render everything fungible) and scientific naturalism (and the attendant pressure to pursue inquiry wherever it logically leads). A sacred value can be defined as any value that a moral community implicitly or explicitly treats as possessing infinite or transcendental significance that precludes comparisons, trade-offs, or indeed any other mingling with bounded or secular values.1 When sacred values are under assault, the apposite functionalist metaphor quickly becomes the intuitive moralist-theologian metaphor,2which depicts people engaged in a continual struggle to protect their private selves and public identities from moral contamination by impure thoughts and deeds (Belk, Wallendorf, & Sherry, 1989). The most emphatic ways to distance oneself from normative transgressions are by (a) expressing moral outrage—a composite psychological state that subsumes cognitive reactions (harsh character attributions to those who endorse the proscribed thoughts and even to those who do not endorse, but do tolerate, this way of thinking in others), affective reactions (anger and contempt for those who endorse the proscribed thoughts), and behavioral reactions (support for ostracizing and punishing deviant thinkers); and (b) engaging in moral cleansing that reaffirms core values and loyalties by acting in ways that shore up those aspects of the moral order that have been undercut by the transgression. Within this framework, rigidity, accompanied by righteous indignation and by blanket refusal even to contemplate certain thoughts, can be commendable—indeed, it is essential for resolutely reasserting the identification of self with the collective moral order (cf. Durkheim,1925/1976). What looks irrationally obdurate within the intuitive scientist and economist research programs can often be plausibly construed as the principled defense of sacred values within the moralist-theologian research program (Tetlock, 1999).
Thus, when (for instance) Charles Murray goes some university to present something, we see strong moral outrage, including against the host of the event (who tolerate his transgressions), and reviewed commitments to racial equality to show their own moral commitment.
A particularly important point made is:
The base-rate literature is both enormous and enormously controversial (Koehler, 1996). Our goal is not, however, just to add to the already formidable list of moderators of whether, and to what degree, people use base rates. Rather, it is to demonstrate that relying on error-prone heuristics is not the only pathway to base-rate neglect. In many contexts, accuracy is neither the only nor even the primary standard for evaluating quality of judgment. A classic example is the U.S. legal system in which procedural justice trumps judgmental accuracy whenever, as often occurs,diagnostic evidence is excluded from trial. Indeed, in exactly this vein, prominent legal theorists have proposed that base-rate evidence is fundamentally inconsistent with the legal ideal of individual justice and should be categorically excluded (Tribe, 1971).
Forbidden base rates refer to any statistical generalization that devoted Bayesians would not hesitate to enter into their probability calculations but that deeply offends a religious or political community. The primary obstacle to using the putatively relevant base rate is not cognitive, but moral. In a society committed to racial,ethnic, and gender egalitarianism, forbidden base rates include observations bearing on the disproportionately high crime rates and low educational test scores of certain categories of human beings. Putting the accuracy and interpretation of such generalizations to the side, people who use these base rates in judging individuals are less likely to be applauded for their skills as good intuitive statisticians than they are to be condemned for their racial and gender insensitivity.
So, for instance, a committed Bayesian rationalist would have to accept that it requires less evidence to convict a man than a woman for a given crime that men commit disproportionately often (which is basically every type of crime except a few special cases such as lying about being rape). The same holds true for convicting blacks on less grounds than whites or Asians, or lower SES people over higher SES people (unless it’s financial crimes!). If one made this base rate explicit in courts — e.g. fingerprints not needed to prove shootings if suspect is black — it would quite obviously offend most people. The results of not doing this would be basically to allow men/blacks/low SES to commit more crimes than they would otherwise be able to — it’s affirmative action of the courtroom.
Also, a particularly hilarious reply to this sort of thinking:
