Evaluation, evaluation, evaluation...

This post was initially published on the Office for Students blog here. As the latest evaluation of the Uni Connect programme is published, John Blake, OfS Director for Fair Access and Participation, highlights the crucial importance of evidence-based practice.

Recently, when I spoke to colleagues from higher education and the third sector about my aspirations for my time as Director for Fair Access and Participation, I said my first priority was ‘evaluation, evaluation, evaluation’.

I acknowledge that this is not a turn of phrase to make the heart flutter, at least not outside the small (but keen to be larger!) community of research nerds who love nothing more than a dive into data and the aroma of analysis. But it is, I firmly believe, the keystone of the bridge of improving not just the work done on access and participation, but forging and reforging the discipline of widening participation theory and practice. Frankly, for 20 years’ worth of essentially compulsory access spending, we do not have 20 years’ worth of evidence about what works.

Of course, that is not to say we have no evidence – there are pockets of exceptional practice, undertaken strategically by widening participation teams and within the OfS-funded Uni Connect partnerships with a powerful influence on work across their institution, and there is a remarkable amount of theoretical discussion of the purpose and process of access and participation work.

But whilst there are places where widening participation theory and practice may meet, it is all-too-often in the manner of Celia Johnson and Trevor Howard in Brief Encounter, aware there is something powerful between them but not quite able to make the longer-term connection. (For completeness, I have an extended version of this metaphor, exploring who and what in the higher education sector are playing the roles of the leads’ respective spouses, the express train and the bit of grit in the eye, but space and readers’ patience prevents further elucidation here.)

I am familiar with this situation from my time in schools – not long after I entered the teaching profession, the advent of social media and blogging blew apart the cosy consensus on school research. Teachers at the chalkface were interrogating the demands of inspectorates, advisors and school leaders and asking what evidence lay behind their preferred pedagogies, curricula and pastoral methods. Teachers were insistent that effective, sustainable practice in the classroom had to be a crucial part of the test of research validity. Theories which provided no useful guidance to those working day-to-day with students, or which recommended activities that took more hours than were available in the week, could not stand.

The result was a turbulent debate, which killed off more than one sacred cow, and system leaders often found it hard to be challenged by relatively junior staff who had availed themselves of evidence previously ignored. But in time a new ecosystem emerged – new providers of schooling entered the system who both rooted their practice in evidence and fed back the results of their activities to drive new research. Ofsted’s inspection framework and the government’s requirements for teacher training were reformed to be explicitly built not on a preferred style of teaching, but on clear reference to the best available evidence. Crucially, to avoid replacing one unquestionable orthodoxy with another, these new models were accompanied by a clear commitment that the evidence would continue to be reviewed, and the frameworks changed to reflect stronger evidence if that arose. Grassroots movements of teachers emerged, determined to keep the discussion of evidence and the generation of research on practice in the forefront of the profession’s mind.

It became essential to the structure of schooling in England that school improvement models be research-based, and that not just system leaders and policymakers, but teaching and other frontline staff, be familiar with the evidence underpinning their methods of working. And, in turn, questions of how we might improve effective practice in the classroom are becoming the key questions for researchers, with crucial work on how to improve the quality and consistency of implementation of interventions, rather than despairing analyses of socioeconomic disparity concluding all is already lost. Fewer hymns to overthrowing the whole system, and more demonstration of the most effective way to structure a maths curriculum to ensure all young people are numerate.

That is what I mean by an ecosystem of evidence-based practice. Those at the frontline do not have to themselves be researchers but need to understand what evidence suggests is best practice and be willing to feed back on their own work. That feedback should go to researchers who are keen to identify and improve best practice, and write with an audience of practitioners in mind. Institutional leaders need to ensure that those involved in widening participation have the clout within the organisation to change direction where the research suggests it is needed, and build the partnerships inside the provider and out which allow the work to be done. Everyone must be open to the possibility that favoured interventions may prove not to be effective, and that activity perhaps previously seen as undesirable, may be more useful.

Of course, the OfS is also part of this ecosystem too, and in that spirit, we are publishing the latest evaluation of our Uni Connect partnership programme. It is clear from the review that partnerships are delivering a huge amount of useful and effective outreach and evaluating their activities. In some cases, activity has not resulted in the improvements we hoped – but that itself is useful in helping us identify future interventions.

Whilst I hope colleagues across the sector will find time to read these reviews and reflect on them, I hope there is also a chance to think more widely about how we build the ecosystem of evidence-based practice we need. At the OfS, we will continue championing evidence-based practice, for our own work, the work we fund, and the work we regulate.

As we reform our approach to access and participation, we want to see more higher education providers developing and sharing high-quality evidence, and more practitioners plugged directly into useful and practical research to enhance their effectiveness. This will help ensure all those considering higher education get the best possible support, advice and intervention to achieve their aspirations.