Home > Scholarly Communication > Journal Peer Review for Preprint Repositories: The Overlay Journal & the RIOJA project

Journal Peer Review for Preprint Repositories: The Overlay Journal & the RIOJA project

Motivation: My interest in scholarly communication has brought me to the concept of “Overlay Journals” and their role in certifying unrefereed research papers (preprints) in preprint repositories. I cover its function and its (to me visible) added value to scholarly communication. Problem statement: Hmm, well if I have to define a problem, I would have to say perhaps my current lack of news and understanding of this concept? Fortunately, I intent to solve this problem with this blog post (and the digging it took to write it).
Findings: Overlay journals are a cost effective way to run a journal. However, there could be difficulties with raising the awareness of your prestige, if you use repositories as your sole base for peer reviewed literature.
Conclusion: Preprint repositories, combined with new initiatives such as the overlay journals, can contribute more and more value to the scientific communities.

Scholars like the concept of having unrefereed research papers (preprints) available to them. The growing use of and content in preprint repositories are indications of that point. This is particularly true in the fields of physics, economics and math, but other fields are growing into this “custom” of sharing preprints as well.

* A preprint is any version prior to peer review and publication, usually the version submitted to a journal.

From “Open Access Overview” by Peter Suber

One significant problem with preprints and their repositories is that, by definition, the preprints are either unrefereed or refereed but not qualified for a publication in some journal (yet). OK, assuming that most of them are largely correct anyhow, they provide value by communicating their contents much earlier than through the journal (peer review) system. And a lot of them will eventually be published by a journal, albeit often with its contents partly modified/improved. However, whether through the formal journal peer review system or individual scrutiny by peers without publication: certification of research papers is always done to guarantee a degree of quality and validity. If only for that extra sense of security that the material being read is valid. And the task of providing that extra sense of security rests on the shoulders of the journal publishers. Conveniently, there is actually a type of journal that solely focuses on treating preprints in repositories as submissions and submit them through the journal peer review process: the overlay journal.

Overlay Journal.
An open-access journal that takes submissions from the preprints deposited at an archive (perhaps at the author’s initiative), and subjects them to peer review. If approved (perhaps after revision), the postprints are also deposited in an archive with some indication that they have been approved. One such indication would be a new citation that included the name of the journal. Another could be a link from the journal’s online table of contents. A third could be new metadata associated with the file. An overlay journal might be associated with just one archive or with many. Because an overlay journal doesn’t have its own apparatus for disseminating accepted papers, but uses the pre-existing system of interoperable archives, it is a minimalist journal that only performs peer review. It is important to Free Online Scholarship (FOS) as an especially low-investment, easily-launched form of open-access journal.

From Guide to the Open Access Movement by Peter Suber

OK, so the differences are not actually that huge when you think about it, because authors who have preprints deposited in preprint repositories commonly submit them to journals for peer review & publication as well. However, for the overlay journals that focus solely on providing journal peer review without actually having their “own place” for “publications”, well then it certainly is a pretty interesting difference. If popular, this overlay journal/preprint repository combination will really put a foot down in the whole “library as publisher” thing (something mentioned in the blog post before this one). And it actually seems to be a lot like what Green/Gold Open Access is essentially striving for: peer review by journal publishers but archived in OA repositories.

I first heard about this overlay journal concept from Peter Suber in an e-mail exchange some time ago, and he told me about the Repository Interface for Overlaid Journal Archives (RIOJA) project. The RIOJA project investigates technical, social and economic aspects of the overlay of quality assurance onto papers deposited to and stored in eprints repositories. And they have been making some pretty interesting progress, which is going to be a significant step in the right direction for not only Open Access but scholarly communication in general as well!

The latest update is the complete report of the results of a survey they have carried out, titled “Repository Interface for Overlaid Journal Archives: results from an online questionnaire survey”:

The RIOJA project will create an interoperability toolkit to enable the overlay of certification onto papers housed in subject repositories. To inform and shape the project, a survey of Astrophysics and Cosmology researchers has been conducted. The findings from that survey form the basis of this report.

Going through the survey, first a small comment on something written on page 3

It is clear that arXiv provides three of the four “first order” functions of a journal, which have been identified[1] as follows

And then it references to a paper by Prosser, David C. (2005) “Fulfilling the promise of scholarly communication – a comparison between old and new access models”. Now, I recall reading and liking this paper myself. However, the original reference concerning the functions of a journal (a scholarly communication model) is Roosendaal, Hans E. and Peter A. Th. M. Geurts (1997). “Forces and functions in scientific communication: an analysis of their interplay”. And referencing the original source is important, as it contributes to maintaining the accuracy and thus the significance of the citation count as a quality filter for research papers.

“Maybe they figured that by referring to this paper readers will automatically find the other paper as well, hitting two birds with one stone (reference)!”

I wonder about that. It could work in practice I guess, but it sounds rather risky. I think it is still better to go with the direct approach. Anyhow, the complete report is 105 pages long (at least the PDF is), so I will start with the ‘Summary of significant observations’ and see how far I can go through the entire survey myself.

“Drama queen. The actual report is only 47 pages, the rest are references and appendices.”

Hehe, well maybe the actual questions in the survey also reveal interesting things. Anyway, moving to 2.2 Publishing your research, page 3:

The average number of papers produced by a scientist in astrophysics and cosmology over a period of 2 years is 13.

On average, that is one paper every two months! Very nice. I wonder if these are the fastest fields in terms of paper production? Also interesting to know would be their average publication %? I mean, depending on that, this result could prove to be even more or less impressive. On the other hand, the focus of the story is on overlay journals and the use of eprint repositories, so I guess this point is not all that relevant to it.

Concerning the acceptance of new models, page 7:

Concerns were expressed about new and untested models of publishing, the overlay model included. However, the respondents were comfortable with the idea of trying new models and means for publishing of scientific research – provided that it could be ensured that the published research outcomes would be eligible for helping to establish an academic record, for attracting funding and ensure tenure. The following issues received particular mention:

  • Impact, readership, sustainability.
  • The peer review process, with particular emphasis on ensuring quality.
  • Open access, repositories and long term archiving,
  • Clarity and proof of viability of the proposed model.

Without a doubt: accreditation is an important incentive for scholars to do their scholarly things. On closer inspection, I wonder if this displayed order is a coincidence or a reflection of the answers of the survey? Like, going by order of importance? I mean, I can see why (financial) sustainability is very important but accreditation above quality assurance? If indeed by order of importance, then that would have been very interesting πŸ™‚

“Not to mention highly controversial.”

After digging through their ‘3.6 Other comments’ section, I have no reason to believe that the order was based on importance or anything like that. Course, this may prove to be true after all if I start digging through the entire comments section in the appendices and try to classify each of them in these groups, but I do not see any leads to do that πŸ™‚

“Lazy bum.”

Ok, I glanced over the open comments on question ’26. If you would like to add something to this survey or have any further comments, please let us know’, where the comments start on page 93 by the way. I have no reason to believe that scholars value accreditation more than quality assurance, as quality/ peer review is mentioned in most of the comments. Anyway, more on the openness of scholars for newer models on page 8 concerning miscellaneous comments:

An area of concern that was repeatedly mentioned in the respondents’ comments was whether there is a need or even a market for a new journal – irrespective of publishing model – in astrophysics and cosmology. On the other hand, the reportedly substantial use of arXiv, and the fact that the vast majority of the respondents use arXiv to get the full text of a paper, suggests that there may be grounds for further exploration of whether a more efficient and speedy way of publishing quality-assured scientific research might be introduced.

Overall I think this openness to new models to improve scholarly communication is a good thing. Particularly when there it is related to (pr)eprint repositories, given their growing significance (to Open Access) and all. I wonder what arXiv thinks of this, though. Will they experiment (more) with features to support the certification of unrefereed research papers in their archive? Hmm, I hope they get to read it. It’s a lot of food for thought.

  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: