{"id":3694,"date":"2013-02-17T23:25:34","date_gmt":"2013-02-17T22:25:34","guid":{"rendered":"http:\/\/emilkirkegaard.dk\/en\/?p=3694"},"modified":"2013-02-17T23:26:22","modified_gmt":"2013-02-17T22:26:22","slug":"useless-lie-detectors-and-wasted-money-the-cost-of-irrationality","status":"publish","type":"post","link":"https:\/\/emilkirkegaard.dk\/en\/2013\/02\/useless-lie-detectors-and-wasted-money-the-cost-of-irrationality\/","title":{"rendered":"Useless lie detectors and wasted money: the cost of irrationality"},"content":{"rendered":"

Eriksson, Anders, and Francisco Lacerda. “Charlatanry in forensic speech science: A problem to be taken seriously.<\/a>” International Journal of Speech, Language and the Law<\/em> 14.2 (2007): 169-193.<\/p>\n

 <\/p>\n

I came across this study while doing research on false rape accusations<\/a>… yes i get around. :P<\/p>\n

 <\/p>\n

https:\/\/en.wikipedia.org\/wiki\/Lie_detection#General_accuracy_and_limitations_of_assessments<\/a><\/p>\n

 <\/p>\n

<\/a><\/a><\/a> In the peer-reviewed academic article “Charlatanry in forensic speech science”, the authors reviewed 50 years of lie detector research and came to the conclusion that there is no scientific evidence supporting that lie detectors actually work.[24]<\/a> Lie detector manufacturer Nemesysco<\/a> threatened to sue the academic publisher for libel resulting in removal of the article from online databases. In a letter to the publisher, Nemesysco’s lawyers wrote that the authors of the article could be sued for defamation if they wrote on the subject again.[25]<\/a>[26]<\/a><\/span><\/p>\n

 <\/p>\n

–<\/p>\n

 <\/p>\n

Abstract<\/strong><\/span><\/p>\n

A lie detector which can reveal lie and deception in some automatic and perfectly <\/span><\/p>\n

reliable way is an old idea we have often met with in science fiction books and comic <\/span><\/p>\n

strips. This is all very well. It is when machines claimed to be lie detectors appear <\/span><\/p>\n

in the context of criminal investigations or security applications that we need to be <\/span><\/p>\n

concerned. In the present paper we will describe two types of \u2018deception\u2019 or \u2018stress <\/span><\/p>\n

detectors\u2019 (euphemisms to refer to what quite clearly are known as \u2018lie detectors\u2019). Both <\/span><\/p>\n

types of detection are claimed to be based on voice analysis but we found no scientific <\/span><\/p>\n

evidence to support the manufacturers\u2019 claims. Indeed, our review of scientific studies <\/span><\/p>\n

will show that these machines perform at chance level when tested for reliability. Given <\/span><\/p>\n

such results and the absence of scientific support for the underlying principles it is <\/span><\/p>\n

justified to view the use of these machines as charlatanry and we argue that there <\/span><\/p>\n

are serious ethical and security reasons to demand that responsible authorities and <\/span><\/p>\n

institutions should not get involved in such practices.<\/span><\/p>\n

keywords: lie detector, charlatanry, voice stress analysis, psychological <\/span><\/p>\n

stress evaluator, microtremor, layered voice analysis, airport <\/span><\/p>\n

security<\/span><\/p>\n

 <\/p>\n

–<\/span><\/p>\n

 <\/p>\n

To keep this distinction in mind has methodological implications. It seems <\/span><\/p>\n

reasonable, from a methodological point of view, to begin by determining <\/span><\/p>\n

the validity of a suggested method before it makes much sense to study its <\/span><\/p>\n

reliability. If the method can be shown to lack validity altogether it will as a <\/span><\/p>\n

consequence also be unreliable and carrying out a reliability test meaningless. <\/span><\/p>\n

If the validity is not known it will be a \u2018black box\u2019 whose reliability, if any, <\/span><\/p>\n

will remain unexplained. We must keep in mind, however, that validity and <\/span><\/p>\n

reliability are not all or nothing concepts. A method may be valid to a degree <\/span><\/p>\n

and reliability may range from very poor to almost perfect. At the far end of <\/span><\/p>\n

the negative scale we find things like astrology. It would be a complete waste of <\/span><\/p>\n

time to design experiments to determine how precisely horoscopes may predict <\/span><\/p>\n

future events when we know that the validity of the method is non-existent. At <\/span><\/p>\n

the positive end of the scale we find methods like DNA testing whose validity <\/span><\/p>\n

is solidly supported by scientific evidence and whose reliability is extremely <\/span><\/p>\n

high, albeit not perfect.<\/span><\/p>\n

 <\/p>\n

they got the implication wrong. the implication is (\u2200x)\u00acreliable(x)\u2192\u00acvalid(x). If a test does not give consistent results for the same stimuli, it cannot possibly be measuring anything. one can infer (contraposition) that (\u2200x)valid(x)\u2192reliable(x). if a test can be used to predict something, its measurements cannot be random. random input cannot predict anything, not even random output. more technically, (\u2200x)(\u2200n)valid(x, n)\u2192(\u2203m)reliable(x, m)\u2227m\u2265n. for any given test and any given number [0-1], if the test has a validity for n for predicting something, it has a reliability of m which is larger than or equal to n. the reliability puts the upper bound on the validity of a test for something.<\/p>\n

 <\/p>\n

In fact astrology has both low to zero reliability* and almost zero validity. for instance, if used in a cohort to predict the height of the person, it can work for the simple reason that children with certain star signs are taller than children with certain other star signs given that they are born the same year. this has nothing to do with astrology, and everything to do with some children having more time to grow than others. if one used a non-children cohert, astrology’s validity goes to 0 again for height prediction. at least, in modern societies where food intake is not dependent on the seasons. for societies where it is, astrology might have some validity, but not becus of planets having any effect on people, just becus the seasons have some effect on people.<\/p>\n

 <\/p>\n

that is, reliability in assigning star signs to people based on a description of their personality, or reliability in giving predictions for the future for any given star sign. for proof of this latter point, find a number of randomly chosen astrology sources, and compare their predictions for the future for any given star sign. they dont agree at all. hence, they as a group cannot possibly predict anything.<\/p>\n

 <\/p>\n

–<\/p>\n

 <\/p>\n

Is there anything we can do to prevent charlatanry in forensic speech <\/strong><\/span><\/p>\n

science?<\/strong><\/span><\/p>\n

Charlatanry, fraud, prejudice and superstition have always been with us. If we <\/span><\/p>\n

look back in history and compare with what we see today there is little that <\/span><\/p>\n

gives us hope that progress in science will diminish the amount of supersti-<\/span><\/p>\n

tious nonsense we see around us. Astrology, for example, seems to be more <\/span><\/p>\n

popular than ever and totally unaffected by how many times astronomers <\/span><\/p>\n

explain that it is complete nonsense. We are therefore somewhat pessimistic <\/span><\/p>\n

about the possibility of efficiently removing charlatanry from forensic speech <\/span><\/p>\n

science. But we hope that responsible authorities like the police and security <\/span><\/p>\n

services will listen to scientifically trained experts in the field rather than to <\/span><\/p>\n

smooth talking and wishful thinking from vendors of bogus lie detectors <\/span><\/p>\n

and similar gadgets. That is probably where we should invest our efforts. We <\/span><\/p>\n

must also take great care when we present our results so that the issue does <\/span><\/p>\n

not appear as a scientific controversy, which it is not. No qualified speech <\/span><\/p>\n

scientist believes in this nonsense so there is absolutely no controversy there, <\/span><\/p>\n

and it is very important that this becomes clear. We have included sufficient <\/span><\/p>\n

detail in this paper to provide the reader with useful arguments in the struggle <\/span><\/p>\n

against charlatanry. We hope that the effort will not turn out to be totally <\/span><\/p>\n

without effect.<\/span><\/p>\n

 <\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Eriksson, Anders, and Francisco Lacerda. “Charlatanry in forensic speech science: A problem to be taken seriously.” International Journal of Speech, Language and the Law 14.2 (2007): 169-193.   I came across this study while doing research on false rape accusations… yes i get around. :P   https:\/\/en.wikipedia.org\/wiki\/Lie_detection#General_accuracy_and_limitations_of_assessments   In the peer-reviewed academic article “Charlatanry in […]<\/p>\n","protected":false},"author":17,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1107],"tags":[],"class_list":["post-3694","post","type-post","status-publish","format-standard","hentry","category-science","entry"],"_links":{"self":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/3694","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/comments?post=3694"}],"version-history":[{"count":2,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/3694\/revisions"}],"predecessor-version":[{"id":3697,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/3694\/revisions\/3697"}],"wp:attachment":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/media?parent=3694"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/categories?post=3694"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/tags?post=3694"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}