To comment or not to comment, that is the question?

Some comments  to Gowers post “Why I’ve also joined the good guys” make me write a third reaction note. I want to understand why there is so much discussion around the idea of  the utility of comments to articles “published” (i.e. selected from arxiv or other free OA repositories) in epijournals.

UPDATE: For epijournals see Episciences.org and also the blog post  Episciences: de quoi s’agit-il?.

UPDATE 2: Read “Comments in epijournals: we may learn from Wikipedia” for a constructive proposal concerning comments (and peer-reviews as well).

I take as examples the comments by Izabella Laba  and  Mike Taylor.  Here they are:

Izabella Laba, link to comment:

I would not submit a paper to a journal that would force me to have a mandatory comment page on every article. I have written several long posts already on this type of issues, so here I’ll only say that this is my well considered opinion based on my decades of experience in mathematics, several years of blogging, and following (and sometimes commenting on) blogs with comment sections of varying quality. No amount of talk about possible fixes etc. will make me change my mind.

Instead, I want to mention a few additional points.

1) A new journal needs to develop a critical mass of authors. While having comment pages for articles may well attract some authors, making them mandatory pages will likely turn off just as many. In particular, the more senior and established authors are less likely to worry about the journal being accepted by promotion committees etc, but also less likely to have the time and inclination to manage and moderate discussion pages.

2) It is tempting to think that every paper would have a lively, engaging and productive comment page. In reality, I expect that this would only happen for a few articles. The majority of papers might get one or two lazy comments. The editors would have to spend time debating whether this or that lazy comment is negative enough or obnoxious enough to be removed, in response to the inevitable requests from the authors; but the point is that no greater good was achieved by having the comment page in the first place.

3) It is also tempting that such comment pages would contain at least a reasonably comprehensive summary of follow-up work (Theorem 1 was extended to a wider class of functions in [A], Conjecture 2 was proved in [B], and the range of exponents in Theorem 3 was proved to be sharp in [C]). But I don’t believe that this will happen. When I write an article, it is my job to explain clearly and informatively how my results relate to existing literature. It is *not* my job to also post explanations of that on multiple comment pages for cited articles, I certainly would not have the time to do that, and I’m not convinced that we could always could on the existence of interested and willing third parties.

A better solution would be to allow pingbacks (say, from the arXiv), so that the article’s journal page shows also the list of articles citing it. Alternatively, authors and editors might be allowed to add post-publication notes of this type (separate from the main article).

4) Related to this, but from a broader perspective: what is it that journals are supposed to accomplish, aside from providing a validation stamp? The old function of disseminating information has already been taken over by the internet. I believe that the most important thing that journals should be doing now is consolidating information, improving the quality of it, raising the signal to noise ratio.

I can see how this goal would be served by having a small number of discussion pages where the commenters are knowledgeable and engaged. In effect, these pages would serve as de facto expository papers in a different format. I do not think that having a large number of comment pages with one or two comments on them would have the same effect. It would not consolidate information – instead, it would diffuse it further.

On a related note, since I mentioned expository papers – it would be excellent to have a section for those. Right now, the journal market for expository papers is very thin: basically, it’s either the Monthly (limited range of topics) or the AMS Bulletin (very small number of papers, each one some sort of a “big deal”). But there is no venue, for instance, for the type of expository papers that researchers often write when they try to understand something themselves. (Except maybe for conference proceedings, but this is not a perfect solution, for many reasons.)

I will likely have more thoughts on it – if so, I’ll post a longer version of this on my own blog.

Mike Taylor, link to comment:

“I would not submit a paper to a journal that would force me to have a mandatory comment page on every article … No amount of talk about possible fixes etc. will make me change my mind.”

I am sorry to hear that. Without in the slighting expecting or intended to change you’re mind, I’ll say this: I can easily imagine that within a few more years, I will be refusing to submit to journals that do not have a comment page on my article. From my perspective, the principle purpose of publishing an article is to catalyse discussion and further work. I am loath to waste my work on venues that discourage this.

“It is tempting to think that every paper would have a lively, engaging and productive comment page. In reality, I expect that this would only happen for a few articles. The majority of papers might get one or two lazy comments.”

The solution to this is probably for us to write more interesting papers.

I totally agree with Mike Taylor and I am tempted to add that authors not willing to accept comments to their articles will deserve a future Darwin award for publication policies.  But surely is their right to lower the chances for their research to  produce descendants.

Say you are a film maker. What do you want?

  • a) to not allow your film to be seen because some of the critics may not appreciate it
  • b) to disseminate your film as much as possible and to learn from the critics and public about eventual weak points and good points of it

If the movie world would be alike to the actual academic world then most of the film makers would choose a), because it does not matter if the film is good or bad, only matters how many films you made and, among them, how many were supported by governmental grants.

A second argument for allowing comments to be made is Wikipedia.  It is clear to (almost) anybody that Wikipedia would not be what it is if it were only based on the 500-1000 regular editors (see the wiki page on Aaron Swartz and Wikipedia). Why is then impossible to imagine that we can make comments to the article a very useful feature of epijournals? Simply by importing some of the well proven rules from wikipedia concerning contributors!

On the reasons of such reactions which disregard the reality, another time. I shall just point to the fact that is still difficult to accept models of thinking based not on pyramidal bureaucratic organizational structures but on massive networked collaboration.   Pre-internet, the pyramidal organization was the most efficient. Post internet it makes no sense because the cost of organizing (Coase cost) went to almost nil.

But thought reflexes are still alive, because we are only humans.

About these ads

2 thoughts on “To comment or not to comment, that is the question?”

  1. or papers, a tool for discussing them (and especially – pointing weak points) would be great. And for me there is not reason _not_ to try it. Especially as I already know a lot of otherwise great papers, which have flaws.

    IMHO the only legitimate concern is about it’s quality. But first, it may be required to provide tag/metadata to comment, depending if it is about spelling, finding a shorter way to proof something, small omission, missing data or a general flaw. (Think – issue tracking software.)

    Then – it can be up/downvoted (even – personally, as in G+), to see which are supported by community, which – not. And if a comment does not apply to standards (e.g. go off-topic, or personal) – then it is community-moderated, as on StackExchange sites.

    When it comes to an argument “because authors does not wish to be commented” – it is ridiculous – like saying ex catherda “I’m infallible, so don’t even try to express doubts”. No. No single individual, even Nobel Prize winner or Fields Medalist (or another No 1 guy in the field) is infallible.

    And authors of comments (and their votes) are not infallible either. :)

  2. When it comes to “Pyramidal bureaucratic organizational structures” there are a few great empirical papers on this phenomenon in academia.

    From my talk http://offtopicarium.wikidot.com/v1:does-2x-bigger-mean-2x-better (BTW: at a totally bottom-up unconferece, Offtopicarium (http://offtopicarium.wikidot.com/en:start)) there is a research: for academic institutions, how does the number of supportive workers scale with number of academic workers. The relation is superlinear (support = c * academic^{1.3} ) – i.e. the more researchers, the higher percentage of supportive workers. Why? The sort story is that once you combine units (e.g. departments into a university) you need to keep the internal administration plus add a new administrative unit for coordination, see the paper: http://dx.doi.org/10.1073/pnas.0903190106.

    But you may say “interesting, but maybe bigger institutes are much more efficient”? Not. There is another study, saying basically that once you fill a critical number of researchers working in a given field, the efficiency per researcher stays constant (and it is from 1-2 for mathematicians to ~20 for experimental physicists, AFAIR). See: http://arxiv.org/abs/1006.0928

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s