Wednesday, October 28, 2009
Thursday, October 08, 2009
At his annual presidential address yesterday afternoon, Ohio State University President E. Gordon Gee thinks it's time for faculty members to be evaluated on the quality and impact of their work.
New faculty members at Ohio State University Libraries enter as assistant professors and have six years to build up their record of scholarship, teaching and service. They receive performance evaluations every year, a fourth-year comprehensive review, and in their sixth year undergo a more vigorous examination to see if they measure up to the level of performance required for tenure and a promotion to associate professor. Library faculty can then choose to undergo an additional review later in the careers to attain the rank of full professor.
President Gee said in his address that professors should be rewarded for their talents and should be encouraged to work with academic departments outside their own. Instead of using an arbitrary formula for evaluation, he would like OSU to create a system in which faculty members are judged on the quality of their work and their impact on students, their disciplines and the community.
This is exactly the position I have been advocating not only on this blog, but in discussions with my library faculty colleagues. Even though I have articulated to colleagues all the points that Gee highlighted, inertia has indeed won out.
From his prepared remarks:
Let me state this directly: We must change our recognition and reward criteria.
Since I returned to Ohio State two years ago, I have made this point a number of times. Changing the way we define scholarship, appreciate new forms of engagement, and properly reward superb teaching can be this University’s signal differential.
If we do not properly and tangibly value those activities, our efforts to extend our resources more fully into our communities will be stymied. We must take it upon ourselves to revise the centuries-old equations for promotion and tenure and develop new reward structures.
Without a doubt, this is a nettlesome issue. And I am not the first person to raise it. Ernie Boyer articulated the case nearly 20 years ago in a speech here on campus. And of course he did so very persuasively in his 1990 book, “Scholarship Reconsidered,” in which he called for “recognition that knowledge is acquired through research, through synthesis, through practice, and through teaching.”
At Ohio State, and at colleges and universities across the country, we have long had faculty committees devoted to looking at revising promotion and tenure standards. And yet, the status quo remains. Inertia is winning.
I believe we must finally speak aloud the truth: that some arbitrary volume of published papers, on some narrowly defined points of debate, is not necessarily more worthy than other activities.
Ladies and gentlemen, this University is big and strong enough to be bold enough to judge by a different standard.
We can dare to say, “No more,” to quantity over quality.
We can stop looking at the length of a vita and start measuring its true heft.
This University, finally, can be the first to say, “We judge by a different standard.” And let others follow our lead, if they wish.
I sit here thinking, what if OSU Libraries HAD acted a year ago and began to change our criteria? Would we have been included in President's speech as the leaders of where the University should be heading? Would that have raised our visibility on campus? As a profession?
So, Thank You! President Gee for validating my, and several of my colleagues, position. Maybe NOW we will be able to break that inertia and finally move ahead. Sphere: Related Content
Wednesday, October 07, 2009
Ms. Fitzpatrick has made the manuscript of Planned Obsolescence: Publishing, Technology, and the Future of the Academy available online for open peer review. The 'book' is a part of Media Commons Press, who's tag line is "open scholarship in open formats."
While the plan is for the manuscript to go through the traditional blind peer-review process, and is forthcoming by NYU Press, Fitzpatrick plans to incorporate reader comments from the online manuscript into her revisions. She asserts:
"One of the points that this text argues hardest about is the need to reform peer review for the digital age, insisting that peer review will be a more productive, more helpful, more transparent, and more effective process if conducted in the open. And so here’s the text, practicing what it preaches, available online for open review."Not only is the process being used to write the manuscript exciting, the manuscript is as well. A couple parts of the text which relate to the academic rewards system:
"our institutional misunderstanding of peer review as a necessary prior indicator of “quality,” rather than as one means among many of assessing quality, dooms us to misunderstand the ways that scholars establish and maintain their reputations within the field."
"we need to remind ourselves, as Cathy Davidson has pointed out, that the materials used in a tenure review are meant in some sense to be metonymic, standing in for the “promise” of all the future work that a scholar will do (“Research”). We currently reduce such “promise” to the existence of a certain quantity of texts; we need instead to shift our focus to active scholarly engagement"
"Until institutional assumptions about how scholarly work should be assessed are changed — but moreover, until we come to understand peer-review as part of an ongoing conversation among scholars rather than a convenient means of determining “value” without all that inconvenient reading and discussion — the processes of evaluation for tenure and promotion are doomed to become a monster that eats its young, trapped in an early twentieth century model of scholarly production that simply no longer works."
"I want to suggest that the time has come for us to consider whether, really, we might all be better served by separating the question of credentialing from the publishing process, by allowing everything through the gate, and by designing a post-publication peer review process that focuses on how a scholarly text should be received rather than whether it should be out there in the first place."Sphere: Related Content
Thursday, October 01, 2009
So suggests a study presented at the Sixth International Congress on Peer Review and Biomedical Publication in Vancouver, Canada, and reported by Nicola Jones in the October 2009 issue of Nature. In his paper "The Natural History of Peer Reviewers: The Decay of Quality" Michael Callaham, editor-in-chief of the Annals of Emergency Medicine in San Francisco, California, reported his analysis of the scores that 84 editors at the journal had been given by nearly 1500 reviewers between 1994 and 2008.
The journal routinely has its editors rate reviews on a scale of one (unsatisfactory) to five (exceptional). The average score stayed at roughly 3.6 throughout the entire period. The surprising result, however, was how individual reviewers' scores changed over time: 93% of them went down, which was balanced by fresh reviewers who kept the average score up. The average decline was 0.04 points per year.
As quoted by Jones, Callaham said "I was hoping some would get better, and I could home in on them. But there weren't enough to study." According to Callaham, less than 1% improved at any significant rate, and even then it would take 25 years for the improvement to become valuable to the journal.
Jones also notes that Callaham agrees that a select few senior advisers are always very useful. But from his own observation, older reviewers do tend to cut corners. Young reviewers assigned a mentor also typically scored half a point better than non-mentored colleagues, but when the mentor's eye disappeared after a year or so, the advantage evaporated. Sphere: Related Content