Does conscientiousness predict PISA scores at the national level? A cautious meta-analysis

Just a quick write-up before I write up a paper with this for ODP.


Altho general cognitive ability (g) has received the most attention by differential psychologists, personality receives a fair share now a days. And just as g has been shown to have great predictive power in large meta-analyses in a variety of contexts (e.g. Gottfredson 1997 is still the best summary IMO), so has the personality trait of conscientiousness (C) (e.g. The Validity of Conscientiousness for Predicting Job Performance A meta-analytic test of two hypotheses A Meta-Analytic Investigation of Conscientiousness in the Prediction of Job Performance Examining the Intercorrelations and the Incremental Validity of Narrow Traits..asp(1) The Case for Conscientiousness Evidence and Implications for a Personality Trait Marker of Health and Longevity).

The ‘new’ thing in differential psych is to study national g estimates and how they correlate. This is the field ive been working mostly in with the spatial transferability hypothesis. The question then is, does C have predictive ability at the national level too? Well, maybe. There are some national estimates of the big five/OCEAN traits in Schmitt et al 2007. I added them to the Megdataset.


Partial correlations

The PISA x measured IQ (not the ones where scholastic ability have been factored in!) correlations were also of interest since no one apparently had calculated the mean PISA x measured IQ correlation. Well, it is .92. So, does C explain some of the remaining variance? One idea is to calculate the partial correlations of C and PISA with mIQ partialed out. However, this method seems to be wrong since some of the correlations are above 1! Ive never seen partial correlations above 1 before.

Math00Mean 1.4828725419
Read00Mean 1.1065080555
Sci00Mean 1.0012991174
Math03Mean 1.0742429148
Read03Mean 1.1147063889
Sci03Mean 1.2609157051
Sci06Mean 0.9137135525
Read06Mean 0.6593605051
Math06Mean 0.3923821506
Read09Mean 0.8607255528
Math09Mean 0.6409903363
Sci09Mean 0.843892485
Finance12Mean 0.3834897092
Math12Mean 0.3682415819
Read12Mean 0.5272534233
Sci12Mean 0.5563931581
CPS12Mean 0.1497008328


Multiple regression

So maybe another method is called for. I used multiple regression on all 17 PISA variables. One may be tempted to simply average them, but as Joost de Winter pointed out to me in an email, the PISA for the same year are not independent. So one cannot just count them as independent. One can get around this problem by doing the meta-analysis within test type, i.e. reading, math and science. Results:

> IQ.betas.weighted.mean
[1] 0.9631086
> C.betas.weighted.mean
[1] 0.1673834
> sum(samples.sizes)
[1] 166Math:
> IQ.betas.weighted.mean
[1] 0.9621924
> C.betas.weighted.mean
[1] 0.02653771
> sum(samples.sizes)
[1] 167Science:
> IQ.betas.weighted.mean
[1] 0.9826468
> C.betas.weighted.mean
[1] 0.1080092
> sum(samples.sizes)
[1] 167

The results from reading have p=.03, so maybe. In 1-2 years, we will have more data from PISA15 to test with. There are plenty of reasons to be cautious: 1) The measured IQs are not perfectly reliably measured. This means that the true correlation between g and PISA scores is higher, leaving less variance to be explained by non-g factors. Maybe nothing? 2) The quality of the personality data is quite poor. Altho one may counter-argue that this is a reason to be more optimistic since the results (well, reading results) are still plausible.

The R sourcecode for the paper is here. The dataset is here.

What about measured IQ and PISA scores?

#the mean PISA x IQ correlation
DF.C.PISA.IQ.rcorr = rcorr(as.matrix(DF.C.PISA.IQ))
IQ.PISA.cors = DF.C.PISA.IQ.rcorr$r[19,] #get IQ row
IQ.PISA.cors = IQ.PISA.cors[2:18] #remove C and IQ-IQ
mean(IQ.PISA.cors) #the mean measured IQ x PISA correlation
#weighted mean
IQ.PISA.cors.n = DF.C.PISA.IQ.rcorr$n[19,] #get IQ row
IQ.PISA.cors.n = IQ.PISA.cors.n[2:18] #remove C and IQ-IQ
IQ.PISA.cors.weighted = IQ.PISA.cors*IQ.PISA.cors.n
IQ.PISA.cors.weighted.mean = sum(IQ.PISA.cors.weighted)/sum(IQ.PISA.cors.n)

The unweighted mean is 0.919, the weighted is 0.924.



Schmitt, D. P., Allik, J., McCrae, R. R., & Benet-Martinez, V. (2007). The Geographic Distribution of Big Five Personality Traits: Patterns and Profiles of Human Self-Description Across 56 Nations. Journal of Cross-Cultural Psychology, 38(2), 173–212. doi:10.1177/0022022106297299

Appendix – full output from MR

PISA test IQ.betas C.betas samples.sizes
Math00Mean 0.9895461 0.096764646 22
Read00Mean 0.977835 0.297191736 22
Sci00Mean 0.9759363 0.099720868 22
Math03Mean 0.9812832 0.016108517 27
Read03Mean 1.0141552 0.27851122 27
Sci03Mean 1.008251 0.104575077 27
Sci06Mean 0.9796918 0.125369373 38
Read06Mean 0.9346129 0.118300942 37
Math06Mean 0.9455623 0.010964361 38
Read09Mean 0.9596431 0.140295939 39
Math09Mean 0.9628133 0.035653129 39
Sci09Mean 0.977768 0.102601624 39
Finance12Mean 0.5286025 -0.144810379 14
Math12Mean 0.9497653 0.001486034 41
Read12Mean 0.9506026 0.094608558 41
Sci12Mean 0.9767656 0.103772057 41
CPS12Mean 0.8830054 -0.025983714 29