{"id":4502,"date":"2014-11-21T19:09:14","date_gmt":"2014-11-21T18:09:14","guid":{"rendered":"http:\/\/emilkirkegaard.dk\/en\/?p=4502"},"modified":"2014-11-21T19:09:14","modified_gmt":"2014-11-21T18:09:14","slug":"installing-scrapy-on-mint-17","status":"publish","type":"post","link":"https:\/\/emilkirkegaard.dk\/en\/2014\/11\/installing-scrapy-on-mint-17\/","title":{"rendered":"Installing scrapy on Mint 17"},"content":{"rendered":"<p>One would think that after so many years, things would simply work on linux. Especially on mainstream platforms. Mint 17 is based on the last Ubuntu LTS version (Trustly Tahr).<\/p>\n<p>For the uninitiated linux can seem difficult with all the terminal commands. But actually install guides make things seem a lot easier than they really are. Case in point, <a href=\"http:\/\/scrapy.org\/\">the scrapy site<\/a> simply lists the command:<\/p>\n<blockquote><p>pip install scrapy<\/p><\/blockquote>\n<p>Very easy right? Well, assuming you already managed to install pip in the first place.<\/p>\n<p>In any case, finding the exact error is not easy. One needs some intuition of where to look for it in the huge log files, and then Google the right error message so one can find the relevant information in the hive-mind (typically stackexchange).<\/p>\n<p><strong>libxml2 and libxslt<\/strong><\/p>\n<p>Buried in the 1000 line log file was an error concerning these:<\/p>\n<blockquote><p>ERROR: \/bin\/sh: 1: xslt-config: not found<br \/>\n** make sure the development packages of libxml2 and libxslt are installed **<\/p><\/blockquote>\n<p>Seems to be solvable by installing these.<\/p>\n<p>sudo apt-get install libxml2-dev<\/p>\n<p>sudo apt-get install libxslt1-dev<\/p>\n<p><strong>libffi<\/strong><\/p>\n<blockquote><p>Package libffi was not found in the pkg-config search path.<br \/>\nPerhaps you should add the directory containing `libffi.pc&#8217;<br \/>\nto the PKG_CONFIG_PATH environment variable<br \/>\nNo package &#8216;libffi&#8217; found<\/p><\/blockquote>\n<p>This was repeated many times. The command to fix is: sudo apt-get install libffi-dev<\/p>\n<p><strong>\/usr\/bin\/ld: cannot find -lz<\/strong><\/p>\n<p><a href=\"http:\/\/stackoverflow.com\/questions\/5178416\/pip-install-lxml-error\">Found solution here<\/a>.<\/p>\n<p>sudo apt-get install -y zlib1g-dev<\/p>\n<p>This error happened when trying to install lxml with pip.<\/p>\n<p><strong>OpenSSL error<\/strong><\/p>\n<p>One error was:<\/p>\n<blockquote><p>cryptography\/hazmat\/bindings\/__pycache__\/_Cryptography_cffi_36a40ff0x2bad1bae.c:194:25: fatal error: openssl\/aes.h: No such file or directory<\/p><\/blockquote>\n<p><a href=\"http:\/\/ask.xmodulo.com\/fix-fatal-error-openssl.html\">Which is discussed here<\/a>. Solution: sudo apt-get install libssl-dev<\/p>\n<p><strong>And the rest<\/strong><\/p>\n<p>The above may not be sufficient. I ran lots of other commands which I have now forgotten. Perhaps some of them made a difference too. Perhaps not.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>One would think that after so many years, things would simply work on linux. Especially on mainstream platforms. Mint 17 is based on the last Ubuntu LTS version (Trustly Tahr). For the uninitiated linux can seem difficult with all the terminal commands. But actually install guides make things seem a lot easier than they really [&hellip;]<\/p>\n","protected":false},"author":17,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[],"tags":[1983,1951,2047],"class_list":["post-4502","post","type-post","status-publish","format-standard","hentry","tag-linux-2","tag-linux","tag-scrapy","entry"],"_links":{"self":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/4502","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/users\/17"}],"replies":[{"embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/comments?post=4502"}],"version-history":[{"count":1,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/4502\/revisions"}],"predecessor-version":[{"id":4503,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/posts\/4502\/revisions\/4503"}],"wp:attachment":[{"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/media?parent=4502"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/categories?post=4502"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/emilkirkegaard.dk\/en\/wp-json\/wp\/v2\/tags?post=4502"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}