Scholarly Communication Metrics Must Change

I read an interesting article by Michael Jensen in the Chronicle entitled the New Metrics of Scholarly Authority a few weeks back, which spawned the stub for this post. Walt Crawford's gray literature column motivated me to finish it.

Scholarly communication before the Internet required the intermediation of publishers. Publishers would go to great lengths to validate the scholarship through peer review. Scholarly authority was then conferred upon those works that were well published, inferred by a scholar's institutional affiliation, or conferred by degrees and tenure status. Scholarly authority was accrued over time in part by the number of references made to their works. Promotion and tenure decisions were made based on these quality indicators.

In an era when the library world changes every 18 months, all too many important library related topics are irrelevant once they go through the glacier like 9-18 month publication cycle. Yet, in the eyes of promotion and tenure committees that paper I wrote in 1994 on Gopher remains to this day more important than this blog - mainly because it was peer reviewed prior to publication and quality indicators can be tracked through ISI Web of Science.

This blog, and the blogs of my colleagues, serves as a very important scholarly communication tool. (I have discussed the merits of scholarly blogging here, here, and here) This blog allows me to get concepts out there as I think of them and receive instant feedback from a qualified network of peers who may, or may not, agree with me. This blog allows any idea I present to be discussed, questioned, and debated upon by a networked peer review community through comments and referrals. Any one of my posting may go through a more thorough post publication review that any one of my print articles. (I also don't have to think of ways to inflate a topic into a 3,ooo word paper.)

We need to begin by widening the definitions of what scholarship and being published are. Promotion and tenure committees must take the time to learn about, and give credit for, the new methods of scholarly communication instead of relying on scholarly publishers as the sole tool in establishing the importance of our contributions.

The problem becomes metrics. How does one quantify the impact that a blog has? The number of subscribers? The number of comments? The number of trackback links? Each has its own set of issues, many not unique to blogging. For example, some have complained about blog cliques which comment and link to each other's posts and in effect boost their individual Technorati rankings. This is not unlike the scholar that references a colleague's work and in effect boosts their citation report. The difference being the turnaround time and ease in which the boosting can be accomplished.

Scholars in all disciplines need to work collectively on ways in which engagement and significance can be measured in the participatory spaces that are putting pressure on the traditional scholarly publication paradigm. Without an agreed upon metric for determining the scholarly value of the biblioblogosphere it appears the traditional authority metrics, and sadly the traditional methods of publication, will remain with us for another decade or so. Sphere: Related Content