The value of (Scientific) Blogs to Scholarly Communication?
Motivation: Exploring the role of the (scientific) blogs in contrast to the traditional ways of scholarly communication.
Problem statement: The role of blogs to support scholarly communication is regularly questioned. Given the strictness of the scientific method, how does the free for all “shotgun” culture of blogs fit in, if at all?
Findings: Blogs to highlight “interesting” scientific findings: suitable, although there are more efficient ways to do so (such as RSS of big scientific news/journal sites and initiatives like Connotea). Then again, as a “new” medium for these kind of things, it is highly suitable. After all, blogs are hot now, and you gotta go with the flow! Blogs to generate scientific communication: suitable, but other initiatives such as forums generally offer better features to support discussions. Blogs to contribute original scientific knowledge: lack of accountability, structure and seemingly effort. It is vastly inferior to preprints, i.e. the scientific paper format, so good luck with that.
Conclusion: We may need something that combines the convenient and efficient services of Connotea with the ease of blogging/forums to efficiently update the scientific folks with the relevant findings and generate discussions.
As I have addressed in an earlier post: the scientific paper format has been designed very efficiently. Journal paper abstracts inform the readers of the topic, the problems, the methodologies, the results, the conclusions and ultimately the added value of the papers in one paragraph. Much like a very short summary of the paper, which is also usually free to read, regardless of whether the paper is Open Access or not. This is done to give readers a short text of the paper so they can decide whether the paper is covering the relevant aspects for them to read in more detail. Afterwards, all these elements, and in particular the methodology, results and conclusion, appear in a more detailed description, so feed the need for information.
This is traditionally missing in blogs, because, they are not an outlet for original research work, but somewhat of an alternative communication channel of already original published works. And even when bloggers blog about peer reviewed/published articles, there is rarely a mention of what is theirs and what is simply from the article but reworded in their own words. That being the case, the added value is then a somewhat personal quality filter with no original scientific added value other than the reference to the original article. And in that case, there are other initiatives that can point scholars to relevant papers that are much more efficient, such as the free online reference management by Nature called Connotea which makes sharing and finding papers rather simple.
Anyway, that scientific blogs traditionally lack (1) a standard format/structure and (2) original scientific information, are rather significant issues to question the usefulness of blogs. Improving the original added value with scientific blogs, while still emphasizing their speedy and easy accessibility, are seemingly difficult to realize, enforce and sustain. I blogged about a standard scientific blog format before, in response to the first issue. However, I am unsure of how appealing such an idea is. Scientific bloggers might not find the idea of adding an extra paragraph (blog abstract if you will) describing their original value (or lack thereof) very interesting. Let us see how some of the research blogging community try to solve the second issue.
BPR3 allows readers to easily find blog posts about serious peer-reviewed research, instead of just news reports and press releases. We provide bloggers with an icon they can use to show when they’re talking about a peer-reviewed work that they’ve read and analyzed closely.
Great idea, I am all for more (optional) quality filters. Assuming it is accurate, it could significantly contribute to the advance of scholarly communication and science in general. However, when going through their guidelines, it seems like they envision doing more than just being a scientific quality filter of blog posts. And there is nothing wrong with trying to add more value to the scientific community, by way of blogs, but I think there are some very complex issues at hand here. Let me address the guidelines that focus on the original value: #4, #5 and #7.
4. The post author should have read and understood the entire work cited.
5. The blog post should report accurately and thoughtfully on the research it presents.
7. The post should contain original work by the post author — while some quoting of others is acceptable, the majority of the post should be the author’s own work.
In theory, I think these are significant guidelines. In practice, I wonder whether they are actually enforceable and sustainable. For one, I have a hard time imaging that their reviewers can actually read every paper/article that a blog references to, and then use that knowledge to verify whether the blog authors themselves understand the scientific works they are linking to. Even if they can right now, it is difficult to maintain sustainability if and when the scientific blog community grows. Additionally, I would have to ask how this process is answerable? Who is responsible for making sure bloggers adhere to these guidelines? What is their experience/background? How do we know they did what they said they would do? How do we measure this?
Assuming it can be enforced and sustained, we could be talking about some pretty significant sacrifices for these activities. I mean, they are solid guidelines. In fact, they are so solid that they are much like the requirements for scholars participating in journal peer reviews. So if they do have the skills to perform these quality assurance activities, is it efficient to use their time evaluating already peer reviewed papers to certify a blog post that may or may not have original added value? Come to think of it, what is considered original value for blog posts anyway? For Research Blogging by BPR3, it includes the rewriting and summarizing of the article in their own words. So added value: yes. New knowledge: no. To be fair, if the blog posts concern errors or other things worthy of concern for published papers, then that would be very meaningful, but just to validate a blog post that highlights how interesting the research is? Or to generate a discussion, which may be more suitable for forums? Or a summary/rewrite of the original article? Is that not a lot of effort for little gain as opposed to, say, peer reviewing unrefereed manuscripts (i.e. preprints) for journal publication, which might truly have some real original knowledge to share with the scientific community?
In fact, if they have to blog & review scientific literature, why not blog & review preprints? That way, they can actually provide original value by contributing to validating the so far unrefereed scientific knowledge. It might even support the journal peer review process. Either directly by submitting these blog peer reviews or after the author has improved the scrutinized manuscript. Much more productive and efficient if you ask me. Of course, reviewing preprints is a bit more challenging (and risky) than just covering peer reviewed publications (i.e. postprints), but if you are going to blog about and scrutinize blogs on scientific knowledge, might as well do it right and focus on stuff that has yet to be validated?
“Well, that is only assuming that the people verifying peer reviewed publications have the right expertise and the time to do the same to unrefereed manuscripts.”
Well, I suppose there is a difference between understanding a scientific article and being able to scrutinize it. I wonder if that is the real issue at hand here?
“I ought to be for you, you’ve never done a formal journal peer review before!”
Well, aside from the fact that I was still not sure what I thought of this initiative on a more serious level, that is why I never bothered to apply for “membership”. However, most of these bloggers that are getting these BPR3 tags do not strike me as people that are unable to perform proper peer reviews. But it is true that it takes significantly less time and effort to cover peer reviewed publications as opposed to peer reviewing preprints. So on a less serious note, blogs do provide that quick and dirty highlights of scientific literature. And since blogs are so popular right now, and probably will stay that for quite some time, I guess there is some advantage of going with the flow to reach out to others? It is hard to make up my mind about this, I guess it requires some more thinking on it.
More on the original value of blogs
Over at RealClimate, a blog post concerning the value of blogs and peer review received quite a few comments. A lot of those posts concerns something about climate physics, and I will not go into that because they confuse the hell out of a non-climate guy like me, but I found this particular comment by Myles Allen rather interesting:
I personally would never comment critically in public on a peer-reviewed paper even to point out “obvious problems” (who is the judge of what is obvious here?) without at least exchanging e-mails with the authors to make sure I had understood it correctly (I’m more than happy to criticize non-peer-reviewed material on Channel 4).
I appreciate that publish-first-and-ask-questions-later is “traditional” practice in blogging, but perhaps, as scientists, we should be challenging that practice.
As far as I am concerned, anything that is published and made publicly available, is free to be criticized. In fact, if there are indeed flaws in it, it should be pointed out for the sake of the other readers and scientific progress in general. However, I also agree that, in terms of scientific papers, that should only be done when you are sure of your case. We would not want it to be a standard practice for mudslinging, reputation smearing, eye gouging dirty fights, after all. And indeed, one way of keeping it civilized while trying to provide value is to contact the authors. Additionally, this could also potentially prevent public embarrassment for both parties. One issue with this measure is that it would significantly slow down or even discourage the concept of criticizing peer reviewed/ published research papers. I mean, what if they wait a long time before responding or simply do not respond at all? And the whole idea of blogs is that it is a fast (and easy) communication medium, and removing that element would remove the key motive for the popularity for blogs I think.
Myles Allen continues this over at Nature’s Climate Feedback blog post on Web 2.0:
Just to be clear, I don’t have a problem with blogging per se, if bloggers were to comply with the old-fashioned courtesy of checking with the authors that they have understood a paper correctly before criticizing it in public (as opposed to over coffee or the conference bar).
If bloggers on high-profile sites like RealClimate were to adopt a simple policy of fact-checking comments on papers with the papers’ authors before posting them, and if necessary posting a response from the authors together with their post, it would certainly be a vast improvement on current practice. The argument that the authors can always respond on the blog doesn’t work, because the responsibility for fact-checking should surely be with the blogger, not his or her unsuspecting targets.
As I agree that preventing is better than curing, I think this is a strong point as well. However, going back to the self-corrective nature of scholarly communication: one can also reason that if the blog is sufficiently popular/significant, the truth will come out one way or the other. Either through other blogs responding to it, or in the comments of that blog post. And if the blog is not popular/significance, then nobody will take notice of it, anyhow. So while risky, it is not an impossible situation to correct. Of course, and this is particularly true for blogs, in between the time of sharing faulty information and the correction, it could have traveled quite far already. Hmm, dilemma.
“What? No closing paragraph to give a sense of closure to this piece?”
I guess I should, but I cannot think of any. Then again, a lack of closure kind of fits this topic, considering its young and dynamic nature. So I guess I will write something extra in the “blog abstract” and forgo writing a “that’s all, folks!” paragraph.