Duc Pham, chance professor of engineering, Department of Mechanical Engineering, University of Birmingham
REF fever is back in earnest. I am referring to the frantic activity in connection with the Research Excellence Framework, the system for assessing the quality of research in UK higher education institutions. As explained in previous CMM articles1, 2, the REF replaced the Research Assessment Exercises, which ran from 1986 until 2008 and were also designed to measure research quality. After years of painstaking preparation, UK universities will soon be entering the final stressful stages of honing their submissions for REF 2021.
An institution has to make a submission for each “unit of assessment”, this being, for example, a department whose research is to be evaluated. The submission comprises three major elements, namely:
- a list of research outputs (predominantly the research publications of the unit);
- a number of impact case studies (examples of the impact of the unit’s research on the economy, society, public policy, culture and quality of life); and
- a narrative about the unit’s research environment (its people, infrastructure and strategy).
There are a few key changes in the upcoming REF from the previous and initial REF in 20143. Perhaps the most radical change is the requirement that institutions return for assessment all their “Category A” or research-active staff, defined as those that have “significant responsibility for research.” Another change is the degree of flexibility in the number of research outputs to be returned per staff member. There is now a minimum of one output to be submitted for each staff member. A maximum of five outputs can be attributed to any staff member, although the total number for a unit must equal 2.5 times the summed full-time equivalent (FTE) of its submitted staff. The weightings for the output, impact and environment scores have been revised to 60, 25 and 15 percent, respectively, to reflect the greater importance placed on research impact. Also, for engineering, there will be one assessing sub-panel rather than separate sub-panels for the different disciplines.
The changes introduced are principally aimed at making the system more transparent and the results more indicative of the true quality of research performed in the UK. Readers may recall that in REF 2014, some institutions were highly selective as to which of their staff to submit for assessment, choosing to present only the work of their best people to optimise their REF outcomes. Thus, a number of institutions succeeded in raising their positions in the league table of REF Grade Point Averages, but the picture of their research could at best be regarded as incomplete. In other words, an institution might have been seen as performing at world-class level in REF 2014, but that was only true of a fraction of its staff and not the whole institution. REF organisers expect the requirement to return all research-active staff for assessment and other new measures in REF 2021 will reduce legitimate game playing by institutions, or will they?
I predict that they will not. With the desire to optimise being a human instinct, people will continually seek clever ways to maximise the return for their efforts. They will adapt to any rules and make them work in their favour. For example, I can see institutions using the new flexibility with the numbers of submitted outputs to increase their output scores. I can also see them moving staff having no research outputs to different categories so that they do not have to be submitted for assessment. These tactics are above board and accord with REF rules, but they will still be deemed game playing by those institutions not practicing them.
As I am playing oracle, let me venture to share a few more predictions to relieve the anxiety of colleagues working on REF 2021 submissions up and down the country, who dread the excruciatingly long wait until December next year to learn the results. Whatever you do, the REF league tables will again be dominated by the Russell Group universities, counting Imperial College London, King’s College London (KCL), London School of Economics and Political Science (LSE), University College London (UCL), University of Cambridge and University of Oxford among those at the top. Pockets of excellence will also emerge at many of the other institutions, hopefully including yours. Overall, the UK will once more see an improvement in the quality of its research. So, there you have them, the results of REF 2021 well in advance of their publication date and at absolutely no cost.
www.birmingham.ac.uk/schools/mechanical-engineering/index.aspx
References
1Pham, D. (2014). The World Cup and Academia. CMM, volume 7, issue 4.
2Pham, D. (2015). Zen and the Art of Excelling in Research Quality Assessment. CMM, volume 8, issue 1, p.35. Available at: http://bit.ly/2vHiv6j
3Guidance on submissions (2019/01) [pdf]. REF. Available at: http://bit.ly/2TmkkOS