Sometimes you need to use a function that wants a numeric matrix as input. One such function is which performs lasso regression with cross validation, which is very cool. Unfortunately, it is picky about how it wants the input data. Here’s some lines of my code:

fit_cv = cv.glmnet(x = as.matrix(temp_df[predictors]), #predictor vars matrix
                   y = as.matrix(temp_df[dependent]), #dep var matrix
                   weights = weights_, #weights
                   alpha = alpha_) #type of shrinkage

We see that x must be a matrix of the predictors, y must be a matrix with the dependent (usually just one), the weights and alpha are optional, but since I am working with aggregate data I am almost always using weights. Alpha controls the kind of shrinkage used.

All well and good, until it isn’t. In my case, the predictor data.frame contains some factor variables. R actually uses numeric values as its internal representation of these, but displays them with strings. For instance:

DF = data.frame(a = 1:3, b = letters[10:12],
                c = seq(as.Date("2004-01-01"), by = "week", len = 3),
                stringsAsFactors = TRUE)

Which prints out like this:

> DF
  a b          c
1 1 j 2004-01-01
2 2 k 2004-01-08
3 3 l 2004-01-15

However, suppose we use my as.matrix solution above, then we get:

> as.matrix(DF)
     a   b   c           
[1,] "1" "j" "2004-01-01"
[2,] "2" "k" "2004-01-08"
[3,] "3" "l" "2004-01-15"

Which is not what we wanted. It gave us a character matrix which will then throw a nonsensical error about. Their bad error made me spend some time finding the actual error. Save yourself and others time. Always write good error messages for functions that will be used more than a couple of times!

Is there some easy built in way to solve the problem?

> as.numeric(DF)
Error: (list) object cannot be coerced to type 'double'

The easiest solution did not work.

> as.numeric(DF$b)
[1] 1 2 3

However, it does work for a single column. So maybe we can just try using it on all the columns:

> apply(DF, 2, as.numeric)
     a  b  c
[1,] 1 NA NA
[2,] 2 NA NA
[3,] 3 NA NA
Warning messages:
1: In apply(DF, 2, as.numeric) : NAs introduced by coercion
2: In apply(DF, 2, as.numeric) : NAs introduced by coercion

What? What is going on?

> apply(as.matrix(DF), 2, as.numeric)
     a  b  c
[1,] 1 NA NA
[2,] 2 NA NA
[3,] 3 NA NA
Warning messages:
1: In apply(as.matrix(DF), 2, as.numeric) : NAs introduced by coercion
2: In apply(as.matrix(DF), 2, as.numeric) : NAs introduced by coercion

It looks apply() does a silent as.matrix() which then causes the NAs. OK. How do we convert just the factor columns then? Maybe try some of the more fancy built in conversion calls:

     a   b   c           
[1,] "1" "j" "2004-01-01"
[2,] "2" "k" "2004-01-08"
[3,] "3" "l" "2004-01-15"


  a b     c
1 1 j 12418
2 2 k 12425
3 3 l 12432

Closer, this time the date got converted, but the factor got converted to character, not integers. We could do a loop:

> for (col_idx in seq_along(DF)) {
+   DF[col_idx] = as.numeric(DF[[col_idx]])
+ }
> DF = as.matrix(DF)
> str(DF)
 num [1:3, 1:3] 1 2 3 1 2 ...
 - attr(*, "dimnames")=List of 2
  ..$ : NULL
  ..$ : chr [1:3] "a" "b" "c"

Which works, but now it is getting silly. Maybe some implicit loops:

> lapply(DF, as.numeric)
[1] 1 2 3
[1] 1 2 3
[1] 12418 12425 12432

Closer, but it returns a list, not a matrix. Maybe just try converting:

> as.matrix(lapply(DF, as.numeric))
a Numeric,3
b Numeric,3
c Numeric,3

But no no, life isn’t that easy. What about

>, as.numeric))
  a b     c
1 1 1 12418
2 2 2 12425
3 3 3 12432

Huh, that works, but as.matrix didn’t. Oh well, just one final step:

> DF = as.matrix(, as.numeric)))
> str(DF)
 num [1:3, 1:3] 1 2 3 1 2 ...
 - attr(*, "dimnames")=List of 2
  ..$ : NULL
  ..$ : chr [1:3] "a" "b" "c"

We got what we wanted!

Sometimes, R does not make your life easy.

In trying to merge some data, I was confronted with a problem of matching up strings where the author had mutilated them. He had done so in two ways: cutting them off at the 8th character or using personal abbreviations. The first one is relatively easy to deal with. The second one is not.

So I looked around a bit and found that there are others who had similar problems:

The large table below shows the matching results. The leftmost column has the strings I need to match. The list to be matched against is the list of country and regional names and their ISO 3 abbreviations here:

It looks like for most cases the shortened version worked fine. 165 of 193 matches were found all of which were correct.

The agrep (with max distance = .1, the default), found a match in 175 cases, so only a little improvement there. But it gets worse, in many cases, it disagrees with the stricter matching method and gets it wrong. There is no case where it is correct over the simpler method. Strangely, in all 10 cases where the simple method failed, agrep got it right. But it got it wrong in the easier cases. In some cases, it is truly bizarre: given the string “United S”, it goes for “United Republic of Tanzania” instead of the much easier “United States”. Strangely, a common error is preferring a subset/longer version over an exact match. No human would make this error. E.g. given “Moldova”, it prefers “Moldova, Republic” of over just “Moldova”.

There are a number of different errors it makes. In the comments below I have noted the type of error (my judgment).

For the moment, I would caution the use of this algorithm.

Country Genetic_distance_SA to_short_result agrep_result best_match agreement filled_in comments
Norway 1455.52 Norway Norway Norway TRUE Norway
Netherla 1453.28 Netherlands Netherlands Netherlands TRUE Netherlands
Ireland 1940.31 Ireland Iceland Ireland FALSE Ireland prefers substitution over exact
Liechste 1511.83 Liechtenstein Liechtenstein FALSE Liechtenstein correct
Germany 1484.92 Germany Germany Germany TRUE Germany
Sweeden 1453.79 Sweden Sweden FALSE Sweden correct
Switzerl 1557.96 Switzerland Switzerland Switzerland TRUE Switzerland
Iceland 1932.36 Iceland Iceland Iceland TRUE Iceland
Denmark 1472.52 Denmark Denmark Denmark TRUE Denmark
Belgium 1940.31 Belgium Belgium Belgium TRUE Belgium
Austria 1465.7 Austria Australia Austria FALSE Austria prefers part deleted
France 1896.22 France France France TRUE France
Slovenia 1292.32 Slovenia Slovenia Slovenia TRUE Slovenia
Finland 2420.3 Finland Finland Finland TRUE Finland
Spain 1929.44 Spain Saint Barth<U+FFFD>lemy Spain FALSE Spain no idea
Italy 1961.9 Italy Italy Italy TRUE Italy
Luxembur 1929.98 TRUE Luxembourg
Czech Re 1524.73 Czech Republic Czech Republic Czech Republic TRUE Czech Republic
U. K. 1916.91 TRUE UK
Greece 1283.05 Greece Greece Greece TRUE Greece
Cyprus 1288.53 Cyprus Cyprus Cyprus TRUE Cyprus
Estonia 2302.86 Estonia Estonia Estonia TRUE Estonia
Slovakia 1573.38 Slovakia Slovakia Slovakia TRUE Slovakia
Malta 1912.52 Malta Gibraltar Malta FALSE Malta prefers subset + substitution over exact
Poland 1905.67 Poland Poland Poland TRUE Poland
Lithuani 2389.28 Lithuania Lithuania Lithuania TRUE Lithuania
Portugal 1949.34 Portugal Portugal Portugal TRUE Portugal
Latvia 2256.69 Latvia Latvia Latvia TRUE Latvia
Croatia 1289.64 Croatia Croatia Croatia TRUE Croatia
Romania 1928.4 Romania Romania Romania TRUE Romania
Bulgaria 1399.01 Bulgaria Bulgaria Bulgaria TRUE Bulgaria
Serbia 1421.01 Serbia Serbia Serbia TRUE Serbia
Russia 1975.49 Russia Russia Russia TRUE Russia
Albania 1301.47 Albania Albania Albania TRUE Albania
Macedoni 1334.51 Macedonia Macedonia Macedonia TRUE Macedonia
Armenia 1558.32 Armenia Armenia Armenia TRUE Armenia
Moldova 1527.95 Moldova Moldova, Republic of Moldova FALSE Moldova prefers longer
Botswana 347.18 Botswana Botswana Botswana TRUE Botswana
South Af 0 South Africa South Africa South Africa TRUE South Africa
Ghana 395.9 Ghana Ghana Ghana TRUE Ghana
Eq Guine 373.2 TRUE Equatorial Guinea
Congo 452.9 Congo Congo Congo TRUE Congo
Kenya 366.78 Kenya Kenya Kenya TRUE Kenya
Cameroon 319.62 Cameroon Cameroon Cameroon TRUE Cameroon
Tanzania 352.54 Tanzania Tanzania Tanzania TRUE Tanzania
Nigeria 342.24 Nigeria Nigeria Nigeria TRUE Nigeria
Uganda 358.75 Uganda Uganda Uganda TRUE Uganda
Zambia 352.54 Zambia Gambia Zambia FALSE Zambia prefers substitution over exact
Sudan 316.95 Sudan Sudan Sudan TRUE Sudan
Zimbabwe 352.54 Zimbabwe Zimbabwe Zimbabwe TRUE Zimbabwe
Ethiopia 705.3 Ethiopia Ethiopia Ethiopia TRUE Ethiopia
Guinea 395.9 Guinea Guinea Guinea TRUE Guinea
CentAfrR 469.7 TRUE Central African Republic
SierraLe 395.9 TRUE Sierra Leone
Mozambiq 355.75 Mozambique Mozambique Mozambique TRUE Mozambique
CongoDR 410.17 Congo Republic of Congo Republic of FALSE Congo Republic of wrong but excuseable; Congo Democratic Republic
Andorra 1912.83 Andorra Andorra Andorra TRUE Andorra
Angola 353.49 Angola Angola Angola TRUE Angola
Belarus 1949.85 Belarus Belarus Belarus TRUE Belarus
Benin 394.54 Benin Benin Benin TRUE Benin
Bosnia 1337.37 Bosnia Bosnia and Herzegovina Bosnia FALSE Bosnia prefers longer
BurkinaF 378.36 Burkina Faso Burkina Faso FALSE Burkina Faso correct
Burundi 362.98 Burundi Burundi Burundi TRUE Burundi
Cape Ver 963.09 Cape Verde Cape Verde Cape Verde TRUE Cape Verde
Chad 537.81 Chad Chad Chad TRUE Chad
Comoros 352.54 Comoros Comoros Comoros TRUE Comoros
IvoryCoa 468.01 TRUE Ivory Coast
Djibouti 750.88 Djibouti Djibouti Djibouti TRUE Djibouti
Eritrea 665.96 Eritrea Eritrea Eritrea TRUE Eritrea
Gabon 360.07 Gabon Gabon Gabon TRUE Gabon
Gambia 395.9 Gambia Gambia Gambia TRUE Gambia
Georgia 1613.7 Georgia Georgia Georgia TRUE Georgia
Guinea-B 395.9 Guinea-Bissau Guinea-Bissau Guinea-Bissau TRUE Guinea-Bissau
Lesotho 352.54 Lesotho Lesotho Lesotho TRUE Lesotho
Liberia 395.9 Liberia Liberia Liberia TRUE Liberia
Malawi 352.54 Malawi Malawi Malawi TRUE Malawi
Mali 430.58 Mali Australia Mali FALSE Mali prefers subset + substitution over exact
Mauritan 681.23 Mauritania Mauritania Mauritania TRUE Mauritania
Namibia 419.32 Namibia Namibia Namibia TRUE Namibia
Niger 315.05 Niger Niger Niger TRUE Niger
Rwanda 364.26 Rwanda Rwanda Rwanda TRUE Rwanda
SaoTomeP 339.89 TRUE Sao Tome and Principe
Senegal 395.9 Senegal Senegal Senegal TRUE Senegal
Seychell 1709.06 Seychelles Seychelles Seychelles TRUE Seychelles
Somalia 500.74 Somalia Somalia Somalia TRUE Somalia
Swazilan 400.17 Swaziland Swaziland Swaziland TRUE Swaziland
Togo 395.9 Togo Togo Togo TRUE Togo
Ukraine 1947.94 Ukraine Ukraine Ukraine TRUE Ukraine
Australi 1971.39 Australia Australia Australia TRUE Australia
United S 1792.33 United States United Republic of Tanzania United States FALSE United States bizarre
New Zeal 2061.12 New Zealand New Zealand New Zealand TRUE New Zealand
Canada 1958.73 Canada Canada Canada TRUE Canada
Japan 2176.04 Japan Japan Japan TRUE Japan
Hong Kon 2674.63 Hong Kong Hong Kong Hong Kong TRUE Hong Kong
Korea 2399.11 Korea Korea Democratic People’s Republic of Korea FALSE Korea prefers subset + substitution over exact
Israel 1539.63 Israel Israel Israel TRUE Israel
Singapor 2459.04 Singapore Singapore Singapore TRUE Singapore
Qatar 1733.83 Qatar Qatar Qatar TRUE Qatar
Hungary 2432.96 Hungary Hungary Hungary TRUE Hungary
Bahrain 971.99 Bahrain Bahrain Bahrain TRUE Bahrain
Chile 2279.52 Chile Chile Chile TRUE Chile
Argentin 1994.59 Argentina Argentina Argentina TRUE Argentina
Barbados 468.02 Barbados Barbados Barbados TRUE Barbados
Uruguay 1918.61 Uruguay Uruguay Uruguay TRUE Uruguay
Cuba 1370.91 Cuba Aruba Cuba FALSE Cuba prefers subset + substitution over exact
Saudi Ar 1468.3 Saudi Arabia Saudi Arabia Saudi Arabia TRUE Saudi Arabia
Mexico 2024.64 Mexico Mexico Mexico TRUE Mexico
Malaysia 1922.77 Malaysia Malaysia Malaysia TRUE Malaysia
Trinidad 1024.1 Trinidad and Tobago Trinidad and Tobago Trinidad and Tobago TRUE Trinidad and Tobago
Kuwait 1081.15 Kuwait Kuwait Kuwait TRUE Kuwait
Lebanon 1543.46 Lebanon Lebanon Lebanon TRUE Lebanon
Venezuel 1280.81 Venezuela, Bolivarian Republic of Venezuela, Bolivarian Republic of Venezuela, Bolivarian Republic of TRUE Venezuela, Bolivarian Republic of
Mauritiu 1792.48 Mauritius Mauritius Mauritius TRUE Mauritius
Jamaica 595.5 Jamaica Jamaica Jamaica TRUE Jamaica
Peru 2096.08 Peru Hviderusland Peru FALSE Peru prefers subset + substitution over exact
Dominica 521.08 Dominica Dominica Dominica TRUE Dominica
SaintLuc 497.7 TRUE Saint Lucia
Ecuador 2228.58 Ecuador Ecuador Ecuador TRUE Ecuador
Brazil 1875.81 Brazil Brazil Brazil TRUE Brazil
SaintVin 395.9 TRUE Saint Vincent
Colombia 1973.6 Colombia Colombia Colombia TRUE Colombia
Iran 1945.07 Iran France Iran FALSE Iran prefers subset + substitution over exact
Tonga 2390.38 Tonga Tonga Tonga TRUE Tonga
Turkey 2167.95 Turkey Turkey Turkey TRUE Turkey
Belize 1481.26 Belize Belize Belize TRUE Belize
Tunisia 203.38 Tunisia Tunisia Tunisia TRUE Tunisia
Jordan 1539.63 Jordan Jordan Jordan TRUE Jordan
SriLanka 1783.84 TRUE Sri Lanka
DomRep 1206.72 TRUE Dominican Republic
W. Samoa 2388.58 W. Samoa W. Samoa W. Samoa TRUE W. Samoa
Fiji 2534.15 Fiji Fiji Fiji TRUE Fiji
China 2646.26 China China China TRUE China
Thailand 2068.81 Thailand Thailand Thailand TRUE Thailand
Surinam 1562.55 Suriname Suriname Suriname TRUE Suriname
Paraguay 2243.61 Paraguay Paraguay Paraguay TRUE Paraguay
Bolivia 2410.22 Bolivia Bolivia, Plurinational State of Bolivia FALSE Bolivia prefers longer
Philipin 2628.84 Philipines Philipines Philipines TRUE Philipines
Egypt 1401.52 Egypt Egypt Egypt TRUE Egypt
Syria 1590.05 Syria Syria Syria TRUE Syria
Honduras 1979.74 Honduras Honduras Honduras TRUE Honduras
Indonesi 2602.63 Indonesia Indonesia Indonesia TRUE Indonesia
VietNam 2264.3 Viet Nam Viet Nam FALSE Viet Nam correct, but odd, vs. Vietnam
Morocco 191.55 Morocco Morocco Morocco TRUE Morocco
Guatemal 2040.41 Guatemala Guatemala Guatemala TRUE Guatemala
Irak 1625.4 Irak Iran Irak FALSE Irak prefers substitution over exact
India 1888.5 India India India TRUE India
Laos 3012.42 Laos Lao People’s Democratic Republic Laos FALSE Laos prefers longer
Pakistan 1901.47 Pakistan Pakistan Pakistan TRUE Pakistan
Madagasc 1678.96 Madagascar Madagascar Madagascar TRUE Madagascar
Papua 3115.88 Papua New Guinea Papua New Guinea FALSE Papua New Guinea correct
Yemen 1190.13 Yemen Yemen Yemen TRUE Yemen
Nepal 2030.11 Nepal Nepal Nepal TRUE Nepal
CookIsla 2437.7 TRUE Cook Islands
Macau 2660.44 Macau Macao Macau FALSE Macau prefers substitution over exact
Marianas 2437.7 Mariana Isl. Mariana Isl. FALSE Mariana Isl. correct
Marshall 2437.7 Marshall Islands Marshall Islands Marshall Islands TRUE Marshall Islands
NCaledon 2437.7 TRUE New Caledonia
Taiwan 2673.71 Taiwan Taiwan, Province of China Taiwan FALSE Taiwan prefers longer
PuertoRi 1654.5 TRUE Puerto Rico
Afghanis 1962.74 Afghanistan Afghanistan Afghanistan TRUE Afghanistan
Algeria 185.65 Algeria Algeria Algeria TRUE Algeria
Antigua/ 491.84 Antigua and Barbuda Antigua and Barbuda FALSE Antigua and Barbuda correct
Azerbaij 2190.35 Azerbaijan Azerbaijan Azerbaijan TRUE Azerbaijan
Bahamas 594.17 Bahamas Bahamas Bahamas TRUE Bahamas
Banglade 1897.24 Bangladesh Bangladesh Bangladesh TRUE Bangladesh
Bhutan 2082.28 Bhutan Bhutan Bhutan TRUE Bhutan
Brunei 1904.48 Brunei Brunei Darussalam Brunei FALSE Brunei prefers longer
Burma 2138.54 Burma Burma Burma TRUE Burma
Cambodia 2254.37 Cambodia Cambodia Cambodia TRUE Cambodia
Costa Ri 1938.1 Costa Rica Costa Rica Costa Rica TRUE Costa Rica
El Salva 1016.14 El Salvador El Salvador El Salvador TRUE El Salvador
Grenada 537.25 Grenada Grenada Grenada TRUE Grenada
Guyana 1379.76 Guyana Guyana Guyana TRUE Guyana
Haiti 434.51 Haiti Haiti Haiti TRUE Haiti
Kazakhst 2122.18 Kazakhstan Kazakhstan Kazakhstan TRUE Kazakhstan
Kiribati 2281.44 Kiribati Kiribati Kiribati TRUE Kiribati
Korea (N 2399.11 Korea North Korea North FALSE Korea North correct
Kyrgysta 2143.13 TRUE Kyrgyzstan
Libya 185.65 Libya Libano Libya FALSE Libya prefers subset, deletion, insertion over exact
Maldives 1836.17 Maldives Maldives Maldives TRUE Maldives
Micrones 2437.7 Micronesia, Federated States of Micronesia, Federated States of Micronesia, Federated States of TRUE Micronesia, Federated States of
Mongolia 2542.15 Mongolia Mongolia Mongolia TRUE Mongolia
Nicaragu 1856.28 Nicaragua Nicaragua Nicaragua TRUE Nicaragua
Oman 1594.25 Oman Cayman Islands Oman FALSE Oman prefers subset + substitution over exact
Panama 1809.12 Panama Panama Panama TRUE Panama
SKittsNe 469.61 TRUE Saint Kitts and Nevis
Solomon 3050.76 Solomon Islands Solomon Islands FALSE Solomon Islands
Tajikist 2000.53 Tajikistan Tajikistan Tajikistan TRUE Tajikistan
TimorLes 2602.63 TRUE Timor–Leste
Turkmeni 2212.49 Turkmenistan Turkmenistan Turkmenistan TRUE Turkmenistan
UArabEm 1286.35 TRUE United Arab Emirates
Uzbekist 2193.47 Uzbekistan Uzbekistan Uzbekistan TRUE Uzbekistan
Vanuatu 2385.83 Vanuatu Vanuatu Vanuatu TRUE Vanuatu


The R code:

gn = read.csv("genetic_distance.csv", encoding = "UTF-8", stringsAsFactors = F)
gn$abbrev = as_abbrev(gn$Country)

trans = read.csv("countrycodes.csv", sep=";", encoding = "UTF-8", stringsAsFactors = F)
trans$shorter = str_sub(trans$Names, 1, 8)

intersect(trans$shorter, gn$Country)

matches = data.frame(source_names = gn$Country,
                     to_short = pmatch (gn$Country, trans$shorter)

agrep(gn$Country[4], trans$shorter)

best_matches = matrix(nrow = nrow(gn))
for (idx in seq_along(gn$Country)) {
  match_idx = agrep(gn$Country[idx], trans$shorter, max.distance = .1, useBytes = T)
  #skip on no match
  if (length(match_idx) == 0) next
  #insert match
  best_matches[idx] = match_idx

matches$agrep = best_matches

matches$to_short_result = trans[matches$to_short, "Names"]
matches$agrep_result = trans[matches$agrep, "Names"]

for (idx in 1:nrow(matches)) {
  if (![idx, "to_short_result"])) {
    matches[idx, "best_match"] = matches[idx, "to_short_result"]
  if ([idx, "to_short_result"])) {
    matches[idx, "best_match"] = matches[idx, "agrep_result"]

write.table(matches, "clipboard", na = "", sep = "\t")

Many airports have free wifi services. The problem with these is that they are time-limited, usually to 1 to 3 hours. This can be very annoying if one is stuck in an airport for an extended period, as I am right now.

Non-technical solution

If you have spent your time on one device, you can simply switch to a new one. If you have brought a smartphone, tablet and a laptop, you can use the time on each of these.

This solution may be sufficient in some situations.

Technical solution

The wifi services rely on your computers MAC address to identity you. They keep track of these and so when you have used all the time on a given MAC, it will be temporarily blocked from using the internet again.

The solution is simple: we kill the batman we switch to a new MAC address every time one has expired. How do we do this? The built in network controller can change the MAC address, but this did not work for me. Instead I downloaded macchanger using:

sudo apt-get install macchanger

This is a small program that lets you easily change MAC addresses. I found a ton of guides, but they did not fully work.

Here’s my current routine.

  1. Disable the wifi using by clicking turn off in the dock-menu.
  2. Delete the previous connection to the network.
  3. Open a terminal as root.
  4. Type:
    macchanger -s wlan0

    to show the current MAC address.

  5. Type:
    macchanger -a wlan0

    to get a new similar MAC address.

  6. Re-do step (4) to see that it worked.
  7. Turn on wifi.
  8. Connect to the network.
  9. Enjoy internet for as long as it lasts, start over from step (1).

I’m not sure if everything here is strictly necessary, but this works for me.

I’ve thought about this question a few times but haven’t found a good solution yet. The problem needs more attention.

AI and computer recognition of images is getting better. I’ll go ahead and say that the probability of reaching the level where computers are as good at humans at any of kind of CAPTCHA/verify-I’m-not-a-bot-test within the next 20 years is very, very high. CAPTCHAs and other newer anti-bot measures cannot continue to work.

We don’t want our comment sections, discussion forums and social media full of spamming bots, so how do we keep them from doing so? I have three proposal types for the online world.

Make sending cost

Generally speaking, spam does not work well. The click-rate is very, very low but it pays because sending mass amounts of it is very cheap. In the analog world, we also see spam in our mail boxes (curiously, this is allowed but online spam is not!), but not quite as much as online. The cost of sending spam in the analog world keeps the amount down (printing and postage).

Participation in the online world is generally free after one has paid one’s internet bill. Generally, stuff that isn’t free on the internet is not used much. Free is the new normal. Free also means the barrier for participation is very low which is good for poor people.

The idea is that we add a cost to writing comments, but keep it very low. Since spam only works when sending large amounts (e.g. 10,000,000 emails per year), and normal human usage does not require sending comparably large amounts (<1,000 emails in most cases), we could add a cost to this which is prohibitively large for bots, but (almost) negligible for humans. E.g. .01 USD per email sent. Thus, human usage would cost <10 USD per year, but botting would cost 100,000 USD.

Who gets the money? One could make it decentralized, so that the blog/newspaper/media owner gets the money. In that way, discussing on a service also supports them, altho very little.

This could maybe work for email spam, but for highly read comment sections (e.g. on major newspapers or official forums for large computer games), the rather small price to pay for writing 1000 (or even 10) posts would not be a deterrent. Hence the pay-for-use proposal fails to deal with some situations.

Making the microtransactions work should not be a problem with cryptocurrencies which can also send anonymously.

Verified users by initial cost

Another idea based on payment is that one can set up a service where users can pay a small fee (e.g. 10 USD) to be registered. The account from this service can then be used to log into the comment section (forum etc.) of other sites and comment for free. The payment can be with cryptocurrency as before so anonymity can be preserved. It is also possible to create multiple outward profiles from one registered account so that a user cannot be tracked from site to site.

If an account has been found to send spam, it can then be disabled and the payment has been wasted. The payment will not have to be large, but it needs to be sufficient to run the service. Perhaps one can outsource the spammer-or-not decision-making to a subset of users who wish to work for free (many services rely upon a subset of users to provide their time, e.g. OKCupid).

The proposal has the same problem as the one above in that it requires payment to participate.

Verified users without payment

A third proposal is to set up a service where one can make a profile for free, but that requires one to somehow prove that one is a real person. This could be done with confidential information about a person e.g. passport + access to a database of this. This would probably require cooperation with officials in each country. Probably they will keep the information about who is who if they can, so it is difficult to see how privacy could be preserved with proposals of this type.

As before, accounts will still need to be deactivated if they are found to be spamming. If the government is involved, they will surely push for other grounds for deactivation: intellectual monopoly infringement, sex work related stuff, foul language, national security matters and so on. This makes it the least preferred solution type to me.

Other solutions?

Generally, the goal is to preserve privacy, no cost of participation and being spam-free. How can it be done? Is online discussion doomed to be overspammed?

So this should be fairly easy according to various guides, but it is not.

The goal is to create a test installation of WordPress on the server, so we can play around with it before moving any changes to the live server.

First step, backup everything!

It is very easy to fuck something up, so you always want to backup stuff. Just in case. To do this, open the FTP and copy all the files. Make sure it was done correctly. One idea is to do it twice and examine the folders. They should be identical (assuming no activity on the site in the meanwhile).

Next you go to the myphpadmin login site. I am using Unoeuro (great host, recommended), so mine is e.g. here:

Go to the database (you can only have one on Unoeuro, it is named after the domain). Then click export, and quick settings.


Copying and fixing the database

WordPress runs of the mySQL database, and two WP installations need their own. To do this, go to the main overview of tables (structure-tab). Then mark the tables WP uses.


Then scroll to the bottom and choose “copy with prefix” in the drop-down menu. Then add the prefix of the tables and the desired prefix of the copies.


You now have a duplicate table set. But wait! In my case, the tables are not copied correctly. The encoding “collation” and type for some reason change to some other stuff. You need to change  them back. To do this, go into each table and choose Opernations-tab. Then change the values so they match the tables from the original WP installation. In my case, to utf8_general_ci and MyISAM.


Do this for each table (there were 11 for me).

Now, we are done? Nope! Next you need to edit the table to tell WP on which domain it is located (for use in relative links presumably). These settings are under wp_options. For some reason there are two identical ones “siteurl” and “home”. These both need to be change to the new URL. In my case from

No edit rights?

However, it wasn’t possible for me to edit them. The error was that no column was set as the uniquely identifying one. To resolve this, go to the SQL-tab and type:


ALTER TABLE `test_journal_OBG_wp_options` ADD UNIQUE (`option_id`)

This sets column “option_id” as the unique column in table “test_journal_OBG_wp_options”. Switch these values with yours. You need to do this for the unique column in every table you want to edit. WP did not make it simple because the unique ID column does not have the same name for each table either, but they are easy to identity.

And then you need to edit the two lines in wp_options.

You do not have sufficient permissions to access this page

Aaaand then, you are not done. When I tried to login, I got an error “You do not have sufficient permissions to access this page.”. Some googling around (1, 2) did not solve the problem for me, but it did suggest the solution to me. The trouble is that within the tables, WP cross-refers to other tables. But these cross-references have not had the prefix added! So you need to find all of them and add it. You need to do that under wp_usermeta and wp_options. I did not find an automatic search and replace, but there are only a few places it needs to be done. Unless you have a lot of users, since you need to do it a few times for every user. Hope you find the research&replace command!

Uploading the copied WordPress

Next step is making a copy of the WordPress folder you want to copy. Then you go to the wp-config.php file and find the line “$table_prefix  = ‘test_journal_OBG_wp_';”. Change this line to the prefix you have chosen and save.

Then upload the WP folder into the correct subfolder on the FTP server and login.

I did not do things in the order above, but it seems the most effective. I may have also forgotten some things I tried. This is just to serve as a reference for those who have similar problems.

Link to article.

Ever been annoyed by the overly long links one gets when one tries to copy a link from Google? Here is an example:

These are very annoying and their function is just to allow Google to track and see which link you actually clicked. Fortunately, there is a way to get rid of them (at least if one is using Firefox which is my favorite browser).

So, after installing GreaseMoney and a suitable script (I tried one first that didn’t work), here is what happens when I copypasta the link again:

Automatic addition of trackers

With the probable closure of The Pirate Bay (TPB) in the near future the torrent scene will be more decentralized. Especially if the TPB tracker goes down. Luckily there are new upcoming trackers to take its place. The problem is that they are not in many of the torrent files. Besides adding the trackers manually there are some ways to fix it. One way is to re-upload all torrents with more trackers. That doesn’t seem like a plausible method. Another way is the have an indexer site (like TPB, Mininova or BTjunkie) update all the torrents with more trackers. This doesn’t seem like a bad idea.

Another idea, and this is my idea for the client, is to have an option in the client to automatically add more trackers to all torrents loaded. That would be very useful for torrents that do not include any new public tracker such as Openbittorrent (OBT)1 or Publicbittorrent (PBT)2. There could be an editable list of trackers to be automatically added, or there could be function that shares all public trackers for all torrents though I’m unsure of what the consequences will be of that. Even more, one could have peers share public trackers. In that way any new public tracker would quickly spread. There is reason to be cautious, however, a third party, say RIAA, may set up such a public tracker just to sniff IPs.

Automatic force-seeding of torrents with no seeds

The goal is to keep rarer torrents alive easier. Currently I have to manually look through my torrent client for torrents that currently have no seeds and then force seed them so that the peers can finish their download.3 It should probably force seed for a set time. If it forced-seeded only until there were at least two seeds, then complications arise. Imagine that there are two people who are the only seeds and both their torrent clients have this feature. Both their clients would then force seed the torrent until they spot another seed, but then they can see each other and so they will keep bouncing on and off force seeding while the peers would not get much data. A mix of these two ideas may be a good idea.


3We’re presuming here for simplicity that if there is no seed, then there is not a complete file in the swarm. This is false in some cases.