In a recent post, Steve Hsu writes:
It’s a shame that we don’t have a better online platform (e.g., like Quora or StackOverflow) for discussing scientific papers. This would allow the authors of a paper to communicate directly with interested readers, immediately after the paper appears.
The best current service I can think of is Reddit. Reddit has tons of subreddits, some of which are concerned with scientific papers. So, for instance, a paper might be posted on r/psychology and get discussion there. However, there is no unified system, so the same paper might get posted several other places independently (this paper has been posted to 5 subreddits as of writing this post). The authors of the paper may not know of the discussion and so will not answer any questions people might have, such as requests for additional analyses which especially relevant when data are not public (most of the time) or analysis is difficult.
One could automatically generate a thread for every published paper. I think this is a bad idea because most papers will get no interest at all (or citations for that matter) and it would necessitate an enormous database. However, one idea is to generate a thread for a paper as soon as a user expresses desire to discuss that paper. This can be done automatically by information extracted from the journal website (metadata, see e.g. Google Scholar’s recommendations), indirectly via the DOI or manually entered (this is however a problem because it opens the door to spammers). Upon generation of the thread and some minimum level of activity in it, one could notify the authors of the paper automatically (using their contact emails in the paper) that someone is discussing it on the site.
The good thing about the internet is that everybody can contribute which is also the bad thing. This means that good contributions can come from anywhere, and lots of useless or counterproductive contributions will also come. The goal is to read the good contributions and to not read the bad contributions. The general solution to this problem is filtering. This can be done both as part of the system (like Reddit’s up and down vote-system), or using client-side scripts. Preferably, the system itself should have multiple ways of filtering the content so that users can pick a filter that gives an output fairly close to their desired output. Reddit has a few options: hot, new, controversial, top, but one could easily add more.
How feasible is such a proposed system? Very feasible, probably even I could set it up if given a bit of time. In practical terms, what would it cost to set it up? One would need to hire some programmers with the relevant expertise (not so cheap) and buy some cloud hosting (fairly cheap).
