{"id":4452,"date":"2014-11-02T21:35:03","date_gmt":"2014-11-02T20:35:03","guid":{"rendered":"http:\/\/emilkirkegaard.dk\/en\/?p=4452"},"modified":"2015-01-09T22:25:55","modified_gmt":"2015-01-09T21:25:55","slug":"w-values-from-the-shapiro-wilk-test-visualized-with-different-datasets","status":"publish","type":"post","link":"https:\/\/emilkirkegaard.dk\/en\/2014\/11\/w-values-from-the-shapiro-wilk-test-visualized-with-different-datasets\/","title":{"rendered":"W values from the Shapiro-Wilk test visualized with different datasets"},"content":{"rendered":"

For a mathematical explanation of the test, see e.g. here<\/a>. However, such an explanation is not very useful for using the test in practice. Just what does a W value of .95 mean? What about .90 or .99? One way to get a feel for it, is to simulate datasets, plot them and calculate the W values. Additionally, one can check the sensitivity of the test, i.e. the p value.<\/p>\n

All the code is in R.<\/p>\n

#random numbers from normal distribution\r\nset.seed(42) #for reproducible numbers\r\nx = rnorm(5000) #generate random numbers from normal dist\r\nhist(x,breaks=50, main=\"Normal distribution, N=5000\") #plot\r\nshapiro.test(x) #SW test\r\n>W = 0.9997, p-value = 0.744<\/pre>\n
\"SW_norm\"<\/a><\/p>\n
So, as expected, W was very close to 1, and p was large. In other words, SW did not reject a normal distribution just because N is large. But maybe it was a freak accident. What if we were to repeat this experiment 1000 times?<\/p>\n
#repeat sampling + test 1000 times\r\nWs = numeric(); Ps = numeric() #empty vectors\r\nfor (n in 1:1000){ #number of simulations\r\n\u00a0 x = rnorm(5000) #generate random numbers from normal dist\r\n\u00a0 sw = shapiro.test(x)\r\n\u00a0 Ws = c(Ws,sw$statistic)\r\n\u00a0 Ps = c(Ps,sw$p.value)\r\n}\r\nhist(Ws,breaks=50) #plot W distribution\r\nhist(Ps,breaks=50) #plot P distribution\r\nsum(Ps<.05) #how many Ps below .05?<\/pre>\n
The number of Ps below .05 was in fact 43, or 4.3%. I ran the code with 100,000 simulations too, which takes 10 minutes or something. The value was 4389, i.e. 4.4%. So it seems that the method used to estimate the P value is slightly off in that the false positive rate is lower than expected.<\/p>\n
What about the W statistic? Is it sensitive to fairly small deviations from normality?<\/p>\n
#random numbers from normal distribution, slight deviation\r\nx = c(rnorm(4900),rnorm(100,2))\r\nhist(x,breaks=50, main=\"Normal distribution N=4900 + normal distribution N=200, mean=2\")\r\nshapiro.test(x)\r\n>W = 0.9965, p-value = 1.484e-09<\/pre>\n
\"\"<\/a>
\nHere I started with a very large norm. dist. and added a small norm dist. to it with a different mean. The difference is hardly visible to the eye, but the P value is very small. The reason is that the large sample size makes it possible to detect even very small deviations from normality. W was again very close to 1, indicating that the distribution was close to normal.<\/p>\n
What about a decidedly non-normal distribution?<\/p>\n
#random numbers between -10 and 10\r\nx = runif(5000, min=-10, max=10)\r\nhist(x,breaks=50,main=\"evenly distributed numbers [-10;10], N=5000\")\r\nshapiro.test(x)\r\n>W = 0.9541, p-value < 2.2e-16<\/pre>\n
 <\/p>\n
\"SW_even\"<\/a><\/p>\n
SW wisely rejects this with great certainty as being normal. However, W is near 1 still (.95). This tells us that the W value does not vary very much even when the distribution is decidedly non-normal. For interpretation then, we should probably bark when W drops just under .99 or so.<\/p>\n
As a further test of the W values, here’s two equal sized distributions plotted together.<\/p>\n
#normal distributions, 2 sd apart (unimodal fat normal distribution)\r\nx = c(rnorm(2500, -1, 1),rnorm(2500, 1, 1))\r\nhist(x,breaks=50,main=\"Mormal distributions, 2 sd apart\")\r\nshapiro.test(x)\r\n>W = 0.9957, p-value = 6.816e-11\r\nsd(x)\r\n>1.436026<\/pre>\n
\"SW_norm3\"<\/a> It still looks fairly normal, altho too fat. The standard deviation is in fact 1.44, or 44% larger than it is supposed to be. The W value is still fairly close to 1, however, and only a little less than from the distribution that was only slightly nonnormal (Ws = .9957 and .9965). What about clearly bimodal distributions?<\/p>\n
#bimodal normal distributions, 4 sd apart\r\nx = c(rnorm(2500, -2, 1),rnorm(2500, 2, 1))\r\nhist(x,breaks=50,main=\"Normal distributions, 4 sd apart\")\r\nshapiro.test(x)\r\n>W = 0.9464, p-value < 2.2e-16<\/pre>\n
\"SW_norm4\"<\/a><\/p>\n
This clearly looks nonnormal. SW rejects it rightly and W is about .95 (W=0.9464). This is a bit lower than for the evenly distributed numbers. (W=0.9541)<\/p>\n
What about an extreme case of nonnormality?<\/p>\n
#bimodal normal distributions, 20 sd apart\r\nx = c(rnorm(2500, -10, 1),rnorm(2500, 10, 1))\r\nhist(x,breaks=50,main=\"Normal distributions, 20 sd apart\")\r\nshapiro.test(x)\r\n>W = 0.7248, p-value < 2.2e-16<\/pre>\n
\"SW_norm5\"<\/a><\/p>\n
Finally we make a big reduction in the W value.<\/p>\n
What about some more moderate deviations from normality?<\/p>\n
#random numbers from normal distribution, moderate deviation\r\nx = c(rnorm(4500),rnorm(500,2))\r\nhist(x,breaks=50, main=\"Normal distribution N=4500 + normal distribution N=500, mean=2\")\r\nshapiro.test(x)\r\n>W = 0.9934, p-value = 1.646e-14<\/pre>\n
\"SW_norm6\"<\/a><\/p>\n
This one has a longer tail on the right side, but it still looks fairly normal. W=.9934.<\/p>\n
#random numbers from normal distribution, large deviation\r\nx = c(rnorm(4000),rnorm(1000,2))\r\nhist(x,breaks=50, main=\"Normal distribution N=4000 + normal distribution N=1000, mean=2\")\r\nshapiro.test(x)\r\n>W = 0.991, p-value < 2.2e-16<\/pre>\n
\"SW_norm7\"<\/a><\/p>\n
This one has a very long right tail. W=.991.<\/p>\n
In conclusion<\/strong><\/p>\n
Generally we see that given a large sample, SW is sensitive to departures from non-normality. If the departure is very small, however, it is not very important.<\/p>\n
We also see that it is hard to reduce the W value even if one deliberately tries. One needs to test extremely non-normal distribution in order for it to fall appreciatively below .99.<\/p>\n","protected":false},"excerpt":{"rendered":"
For a mathematical explanation of the test, see e.g. here. However, such an explanation is not very useful for using the test in practice. Just what does a W value of .95 mean? What about .90 or .99? One way to get a feel for it, is to simulate datasets, plot them and calculate the […]<\/p>\n","protected":false},"author":17,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[2039,1979,2038,1202],"class_list":["post-4452","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-normality","tag-r","tag-shapiro-wilk-test","tag-statistics","entry"],"_links":{"self":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/4452","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/comments?post=4452"}],"version-history":[{"count":4,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/4452\/revisions"}],"predecessor-version":[{"id":4593,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/4452\/revisions\/4593"}],"wp:attachment":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/media?parent=4452"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/categories?post=4452"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/tags?post=4452"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}