As a result of my posts on this blog last year relating to Britain’s Research Excellence Framework (see especially here and here), I was invited to write a short book to inaugurate the new “Sage Swifts” series.
Rank Hypocrisies: The Insult of the REF will be published on December 3, 2014: a couple weeks before HEFCE is due to publish the REF results.
But today’s announcement that HEFCE is actively “exploring the benefits and challenges of expanding … the Research Excellence Framework (REF), on an international basis” with a view to “an extension of the assessment to incorporate submissions from universities overseas” suggests some advance publicity might not be untimely. For the REF should come with a health warning.
What I find most chilling in today’s HEFCE announcement is the bald assertion (in the accompanying survey) that “The UK’s research assessment system has a positive international reputation, built on a methodology developed over more than 20 years.”
Rank Hypocrisies shows on the contrary that the procedures used to evaluate outputs by Britain’s REF panels make a mockery of peer review as understood within the international academic community. Among the issues discussed are the narrow disciplinary remit of REF panels and their inability to evaluate interdisciplinary research, the risks of replication of entrenched academic hierarchies and networks inherent in HEFCE’s procedures for appointment of panel members, the utterly unrealistic volume of work expected of panelists, the perversity of excluding all external indicators of quality from many assessments, and the lack of competence of REF panels to provide sufficient diversity and depth of expertise to evaluate the outputs that fall under their remit.
The REF is a system in which overburdened assessors assign vaguely defined grades in fields that are frequently not their own while (within many panels) ignoring all external indicators of the academic influence of the publications they are appraising, then shred all records of their deliberations. That HEFCE should now be seeking to extend such a “methodology” beyond Britain’s shores is risible.
Derek Sayer’s book is essential reading for all university researchers and research policy makers. It discusses the waste, biases and pointlessness of Britain’s Research Excellence Framework (REF), and its misuse by universities. The book is highly readable, astute, sharply analytical and very intelligent. It paints a devastating portrait of a scheme that is useless for advancing research and that does no better job at ranking research performance than do the global indexes but does so for a huge cost in time, money, duplication, and irritation. Anyone interested in research ranking, assessment, and the contemporary condition of the universities should read this book.
Peter Murphy, Professor of Arts and Society, James Cook University
Rank Hypocrisies offers a compellingly convincing critique of the research auditing exercise to which university institutions have become subject. Derek Sayer lays bare the contradictions involved in the REF and provides a forensic analysis of the problems and inconsistencies inherent in the exercise as it is currently constituted. A must read for all university academic staff and the fast multiplying cadre of higher education managers and, in particular, government ministers and civil servants in the Department of Business Innovation and Skills.
Barry Smart, Professor of Sociology, University of Portsmouth
Academics across the world have come to see the REF – and its RAE predecessor – as an arrogant attempt to raise national research standards that has resulted in a variety of self-inflicted wounds to UK higher education. Derek Sayer is the Thucydides of this situation. A former head of the Lancaster history department, he fell on his sword trying to deal with a university that behaved in an increasingly irrational manner as it tried to game a system that is fundamentally corrupt in both its conception and execution. Rank Hypocrisies is more than a cri de coeur. It is the best documented diagnosis of a regime that has distorted the idea of peer review beyond recognition. Only someone with the clear normative focus of a former insider could have written this work. Thucydides would be proud.”
Steve Fuller, Auguste Comte Chair in Social Epistemology, Warwick University
Sayer makes a compelling argument that the Research Excellence Framework is not only expensive and divisive, but is also deeply flawed as an evaluation exercise. Rank Hypocrisies is a rigorous and scholarly evaluation of the REF, yet written in a lively and engaging style that makes it highly readable.
Dorothy Bishop, Professor of Developmental Neuropsychology and Wellcome Principal Research Fellow, University of Oxford
The REF is right out of Havel’s and Kundera’s Eastern Europe: a state-administered exercise to rank academic research like hotel chains – 2 star, 3 star – dependent on the active collaboration of the UK professoriate. In crystalline text steeped in cold rage, Sayer takes aim at the REF’s central claim, that it is a legitimate process of expert peer review. He provides a short history of the RAE/REF. He critiques university and national-level REF processes against actual practices of scholarly review as found in academic journals, university presses, and North American tenure procedures. His analysis is damning. If the REF fails as scholarly review, how can academics and universities continue to participate? And how can government use its rankings as a basis for public policy?
Tarak Barkawi, Reader in the Department of International Relations, London School of Economics
More details of the book (which will be available in hardback and electronic formats) may be found here.