{"id":4410,"date":"2014-10-13T10:37:22","date_gmt":"2014-10-13T09:37:22","guid":{"rendered":"http:\/\/emilkirkegaard.dk\/en\/?p=4410"},"modified":"2014-10-14T12:17:05","modified_gmt":"2014-10-14T11:17:05","slug":"does-conscientiousness-predict-pisa-scores-at-the-national-level-a-cautious-meta-analysis","status":"publish","type":"post","link":"https:\/\/emilkirkegaard.dk\/en\/2014\/10\/does-conscientiousness-predict-pisa-scores-at-the-national-level-a-cautious-meta-analysis\/","title":{"rendered":"Does conscientiousness predict PISA scores at the national level? A cautious meta-analysis"},"content":{"rendered":"

Just a quick write-up before I write up a paper with this for ODP.<\/em><\/p>\n

Introduction<\/strong><\/p>\n

Altho general cognitive ability (g) has received the most attention by differential psychologists, personality receives a fair share now a days. And just as g has been shown to have great predictive power in large meta-analyses in a variety of contexts (e.g. Gottfredson 1997<\/a> is still the best summary IMO), so has the personality trait of conscientiousness (C) (e.g.\u00a0The Validity of Conscientiousness for Predicting Job Performance A meta-analytic test of two hypotheses<\/a> A Meta-Analytic Investigation of Conscientiousness in the Prediction of Job Performance Examining the Intercorrelations and the Incremental Validity of Narrow Traits..asp(1)<\/a> The Case for Conscientiousness Evidence and Implications for a Personality Trait Marker of Health and Longevity<\/a>).<\/p>\n

The ‘new’ thing in differential psych is to study national g estimates and how they correlate. This is the field ive been working mostly in with the spatial transferability hypothesis. The question then is, does C have predictive ability at the national level too? Well, maybe. There are some national estimates of the big five\/OCEAN traits<\/a> in Schmitt et al 2007. I added them to the Megdataset<\/a>.<\/p>\n

 <\/p>\n

Partial correlations<\/strong><\/p>\n

The PISA x measured IQ (not the ones where scholastic ability have been factored in!) correlations were also of interest since no one apparently had calculated the mean PISA x measured IQ correlation. Well, it is .92. So, does C explain some of the remaining variance? One idea is to calculate the partial correlations of C and PISA with mIQ partialed out. However, this method seems to be wrong since some of the correlations are above 1! Ive never seen partial correlations above 1 before.<\/p>\n\n<\/colgroup>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
<\/td>\nConscientiousnessMeanSchmittEtAl2007<\/td>\n<\/tr>\n
Math00Mean<\/td>\n1.4828725419<\/td>\n<\/tr>\n
Read00Mean<\/td>\n1.1065080555<\/td>\n<\/tr>\n
Sci00Mean<\/td>\n1.0012991174<\/td>\n<\/tr>\n
Math03Mean<\/td>\n1.0742429148<\/td>\n<\/tr>\n
Read03Mean<\/td>\n1.1147063889<\/td>\n<\/tr>\n
Sci03Mean<\/td>\n1.2609157051<\/td>\n<\/tr>\n
Sci06Mean<\/td>\n0.9137135525<\/td>\n<\/tr>\n
Read06Mean<\/td>\n0.6593605051<\/td>\n<\/tr>\n
Math06Mean<\/td>\n0.3923821506<\/td>\n<\/tr>\n
Read09Mean<\/td>\n0.8607255528<\/td>\n<\/tr>\n
Math09Mean<\/td>\n0.6409903363<\/td>\n<\/tr>\n
Sci09Mean<\/td>\n0.843892485<\/td>\n<\/tr>\n
Finance12Mean<\/td>\n0.3834897092<\/td>\n<\/tr>\n
Math12Mean<\/td>\n0.3682415819<\/td>\n<\/tr>\n
Read12Mean<\/td>\n0.5272534233<\/td>\n<\/tr>\n
Sci12Mean<\/td>\n0.5563931581<\/td>\n<\/tr>\n
CPS12Mean<\/td>\n0.1497008328<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

Multiple regression<\/strong><\/p>\n

So maybe another method is called for. I used multiple regression on all 17 PISA variables. One may be tempted to simply average them, but as Joost de Winter pointed out to me in an email, the PISA for the same year are not independent. So one cannot just count them as independent. One can get around this problem by doing the meta-analysis within test type, i.e. reading, math and science. Results:<\/p>\n

\n
Reading:
\n> IQ.betas.weighted.mean
\n[1] 0.9631086
\n> C.betas.weighted.mean
\n[1] 0.1673834
\n> sum(samples.sizes)
\n[1] 166Math:
\n> IQ.betas.weighted.mean
\n[1] 0.9621924
\n> C.betas.weighted.mean
\n[1] 0.02653771
\n> sum(samples.sizes)
\n[1] 167Science:
\n> IQ.betas.weighted.mean
\n[1] 0.9826468
\n> C.betas.weighted.mean
\n[1] 0.1080092
\n> sum(samples.sizes)
\n[1] 167<\/div>\n<\/blockquote>\n

The results from reading have p=.03, so maybe. In 1-2 years, we will have more data from PISA15 to test with. There are plenty of reasons to be cautious: 1) The measured IQs are not perfectly reliably measured. This means that the true correlation between g and PISA scores is higher, leaving less variance to be explained by non-g factors. Maybe nothing? 2) The quality of the personality data is quite poor. Altho one may counter-argue that this is a reason to be more optimistic since the results (well, reading results) are still plausible.<\/p>\n

The R sourcecode for the paper is here<\/a>. The dataset is here<\/a>.<\/p>\n

What about measured IQ and PISA scores?<\/strong><\/p>\n

#the mean PISA x IQ correlation
\nDF.C.PISA.IQ.rcorr = rcorr(as.matrix(DF.C.PISA.IQ))
\nIQ.PISA.cors = DF.C.PISA.IQ.rcorr$r[19,] #get IQ row
\nIQ.PISA.cors = IQ.PISA.cors[2:18] #remove C and IQ-IQ
\nmean(IQ.PISA.cors) #the mean measured IQ x PISA correlation
\nround(IQ.PISA.cors,2)
\n#weighted mean
\nIQ.PISA.cors.n = DF.C.PISA.IQ.rcorr$n[19,] #get IQ row
\nIQ.PISA.cors.n = IQ.PISA.cors.n[2:18] #remove C and IQ-IQ
\nIQ.PISA.cors.weighted = IQ.PISA.cors*IQ.PISA.cors.n
\nIQ.PISA.cors.weighted.mean = sum(IQ.PISA.cors.weighted)\/sum(IQ.PISA.cors.n)<\/p><\/blockquote>\n

The unweighted mean is 0.919, the weighted is 0.924.<\/p>\n

 <\/p>\n

Refs<\/strong><\/p>\n

\n

Schmitt, D. P., Allik, J., McCrae, R. R., & Benet-Martinez, V. (2007). The Geographic Distribution of Big Five Personality Traits: Patterns and Profiles of Human Self-Description Across 56 Nations<\/a>. Journal of Cross-Cultural Psychology<\/i>, 38<\/i>(2), 173\u2013212. doi:10.1177\/0022022106297299<\/p>\n

\n

Appendix – full output from MR<\/strong><\/p>\n

<\/div>\n
\n\n<\/colgroup>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
PISA test<\/span><\/strong><\/td>\n IQ.betas<\/span><\/strong><\/td>\n C.betas<\/span><\/strong><\/td>\nsamples.sizes<\/span><\/strong><\/td>\n<\/tr>\n
Math00Mean<\/span><\/td>\n0.9895461<\/span><\/td>\n0.096764646<\/span><\/td>\n22<\/span><\/td>\n<\/tr>\n
Read00Mean<\/span><\/td>\n0.977835<\/span><\/td>\n0.297191736<\/span><\/td>\n22<\/span><\/td>\n<\/tr>\n
Sci00Mean<\/span><\/td>\n0.9759363<\/span><\/td>\n0.099720868<\/span><\/td>\n22<\/span><\/td>\n<\/tr>\n
Math03Mean<\/span><\/td>\n0.9812832<\/span><\/td>\n0.016108517<\/span><\/td>\n27<\/span><\/td>\n<\/tr>\n
Read03Mean<\/span><\/td>\n1.0141552<\/span><\/td>\n0.27851122<\/span><\/td>\n27<\/span><\/td>\n<\/tr>\n
Sci03Mean<\/span><\/td>\n1.008251<\/span><\/td>\n0.104575077<\/span><\/td>\n27<\/span><\/td>\n<\/tr>\n
Sci06Mean<\/span><\/td>\n0.9796918<\/span><\/td>\n0.125369373<\/span><\/td>\n38<\/span><\/td>\n<\/tr>\n
Read06Mean<\/span><\/td>\n0.9346129<\/span><\/td>\n0.118300942<\/span><\/td>\n37<\/span><\/td>\n<\/tr>\n
Math06Mean<\/span><\/td>\n0.9455623<\/span><\/td>\n0.010964361<\/span><\/td>\n38<\/span><\/td>\n<\/tr>\n
Read09Mean<\/span><\/td>\n0.9596431<\/span><\/td>\n0.140295939<\/span><\/td>\n39<\/span><\/td>\n<\/tr>\n
Math09Mean<\/span><\/td>\n0.9628133<\/span><\/td>\n0.035653129<\/span><\/td>\n39<\/span><\/td>\n<\/tr>\n
Sci09Mean<\/span><\/td>\n0.977768<\/span><\/td>\n0.102601624<\/span><\/td>\n39<\/span><\/td>\n<\/tr>\n
Finance12Mean<\/span><\/td>\n0.5286025<\/span><\/td>\n-0.144810379<\/span><\/td>\n14<\/span><\/td>\n<\/tr>\n
Math12Mean<\/span><\/td>\n0.9497653<\/span><\/td>\n0.001486034<\/span><\/td>\n41<\/span><\/td>\n<\/tr>\n
Read12Mean<\/span><\/td>\n0.9506026<\/span><\/td>\n0.094608558<\/span><\/td>\n41<\/span><\/td>\n<\/tr>\n
Sci12Mean<\/span><\/td>\n0.9767656<\/span><\/td>\n0.103772057<\/span><\/td>\n41<\/span><\/td>\n<\/tr>\n
CPS12Mean<\/span><\/td>\n0.8830054<\/span><\/td>\n-0.025983714<\/span><\/td>\n29<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Just a quick write-up before I write up a paper with this for ODP. Introduction Altho general cognitive ability (g) has received the most attention by differential psychologists, personality receives a fair share now a days. And just as g has been shown to have great predictive power in large meta-analyses in a variety of […]<\/p>\n","protected":false},"author":17,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1839,2591],"tags":[2027,2026],"class_list":["post-4410","post","type-post","status-publish","format-standard","hentry","category-psychometics","category-intelligence-iq-cognitive-ability","tag-conscientiousness","tag-pisa","entry"],"_links":{"self":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/4410","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/comments?post=4410"}],"version-history":[{"count":5,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/4410\/revisions"}],"predecessor-version":[{"id":4422,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/4410\/revisions\/4422"}],"wp:attachment":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/media?parent=4410"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/categories?post=4410"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/tags?post=4410"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}