You are currently viewing Educational interventions keep not working

Educational interventions keep not working

We’ve been over this topic previously in 2017, but since this is the eternal idea, let’s look at more data.

Let’s start historically. 1969 Arthur Jensen’s famous article “How much can we boost IQ and scholastic achievement?

Compensatory education has been tried and it apparently has failed.
Compensatory education has been practiced on a massive scale for several years in many cities across the nation. It began with auspicious enthusiasm and high hopes of educators. It had unprecedented support from Federal funds. It had theoretical sanction from social scientists espousing the major underpinning of its rationale: the “deprivation hypothesis,” according to which academic lag is mainly the result of social, economic, and educational deprivation and dis­crimination— an hypothesis that has met with wide, uncritical acceptance in the atmosphere of society’s growing concern about the plight of minority groups and the economically disadvantaged.

The chief goal of compensatory education— to remedy the educational lag of disadvantaged children and thereby narrow the achievement gap between “minority” and “majority” pupils— has been utterly unrealized in any of the large compensatory education programs that have been evaluated so far. On the basis of a nationwide survey and evaluation of compensatory education programs, the United States Commission on Civil Rights (1967) came to the following conclusion: The Commission’s analysis does not suggest that compensatory education is incapable of remedying the effects of poverty on the academic achievement of individual children. There is little question that school programs involving expenditures for cultural en­richment, better teaching, and other needed educational services can be helpful to dis­ advantaged children. The fact remains, however, that none of the programs appear to have raised significantly the achievement of participating pupils, as a group, within the period evaluated by the Commission, (p. 138)

He relies on several large studies carried out by the government in the 1960s, and which basically found that compensatory education, usually preschool (kindergarten or even before), did not work for its stated goals. Of course, others did not like these conclusions, so there has been another 60 years of such programs. And where are we now? Well…

2010; Head Start Impact Study

In the 1998 reauthorization of Head Start, Congress directed the US Department of Health and Human Services (DHHS) to determine, on a national level, the impact of Head Start on the children it serves. This the first of two reports addressing that question by reporting on the impacts of Head Start on children and families during the children’s preschool, kindergarten, and 1st grade years in two main areas:

1) Key child and parent outcomes, particularly as they apply to children’s school readiness

2) The circumstances under which Head Start might achieve the greatest impact.

The impact study found that providing access to Head Start has a positive impact on children’s preschool experiences. Access to Head Start also has positive impacts on several aspects of children’s school readiness during their time in the program. However, the advantages children gained during their time in Head Start and up to age 4 yielded only a few statistically significant differences in outcomes at the end of 1st grade for the sample as a whole.

So it didn’t work. One can of course find people who go on fishing trips in the data afterwards (this is also true for the similarly negative study, Moving to Opportunity). This always happens and proves nothing more than people can find p < .05 when they try a lot of times. It is important to stick to the planned analyses to avoid the p-hacking.

2013: meta-analysis of US-based interventions to boost scholastic achievement

Obvious and massive publication bias and seemingly declining effect size over the years. As science gets better, results become worse. Metallic laws not surprised.

2018: eternal optimists at left-wing Brookings Institute conclude pre-K doesn’t work

I have argued elsewhere that the policy path forward for the center-based care and education of young children is muddled.[18] The present analysis reinforces that judgment. Putting nearly all our eggs in the same basket — enhancing access to state pre-K for four-year-olds – shows little evidence to date of having a substantive payoff in later school achievement. It is time for enthusiasts for increased investments in state pre-K to confront the evidence that it does not enhance student achievement meaningfully. They need to temper their enthusiasm for more of the same and, instead, support testing of other approaches that appear promising.[19]

2019: meta-analysis of 141 large RCTs

There are a growing number of large-scale educational randomized controlled trials (RCTs). Considering their expense, it is important to reflect on the effectiveness of this approach. We assessed the magnitude and precision of effects found in those large-scale RCTs commissioned by the UK-based Education Endowment Foundation and the U.S.-based National Center for Educational Evaluation and Regional Assistance, which evaluated interventions aimed at improving academic achievement in K–12 (141 RCTs; 1,222,024 students). The mean effect size was 0.06 standard deviations. These sat within relatively large confidence intervals (mean width = 0.30 SDs), which meant that the results were often uninformative (the median Bayes factor was 0.56). We argue that our field needs, as a priority, to understand why educational RCTs often find small and uninformative effects.

Their Bayesian stuff, we don’t necessarily care about, just that the mean effect size was a measly 0.06 d, equivalent to a correlation about 0.03. This sets a very strong near null prior on such interventions. It is also evidence that the differences are genetic because ruling out large chunks of environmental variation as causal increases the probability that remaining variation is causal, which will then be genetic variation. Somehow this implication seems lost on many people.

As I have said a few times: mainstreamers keep making progress towards the inevitable. Not exactly by the speed of funerals, but not exactly fast either. Arthur Jensen’s 1969 position was far out of the mainstream in general, but now a days, most experts seem to agree with him on the main points. We’ve got socialists arguing for behavioral genetics, belief in educational interventions seems falling, and surveys repeatedly show that researchers agree that genetics has substantial causal effect on the black-white intelligence gap in the USA.