Why is evaluation in EdTech needed?

If one trend in the education sector is clear, it is that the EdTech industry is still booming, even after its rapid acceleration instigated by the mass remote-learning that resulted from stay at home orders during the pandemic. But tight school budgets and proliferation of EdTech products in the market mean that evidence on EdTech and its impact on learners is more important than ever for buyers. The UK Government’s Education Technology (EdTech) Survey 2020-21 found that schools’ EdTech buying decisions are frequently influenced by the availability of evaluation data. Teachers and headteachers especially value recommendations from research bodies, leadership, teaching staff, technical staff and other schools specialising in technology. Evaluating EdTech is a powerful way to build trust with schools wanting to invest in EdTech solutions.

The Talking Impact Event

Wanting to draw out some practical advice from both EdTech and evaluation experts on how to evaluate, we held our third Talking Impact workshop to discuss all things evaluation, impact, and evidence on the 21st of June via Zoom. This time, the spotlight was on evaluation in the EdTech sector. We presented our paper synthesising findings from evaluation of EdTech programmes based on some of our partnerships with organisations like the DfE, TextHelp, and Nesta. The paper also provides recommendations for EdTech companies on how to evaluate their products. The paper is published on our website here.

We were grateful to have three brilliant panellists for the event - James from United Learning, Patrick McGrath from TextHelp, and our very own Kristy Evers, Director of Impact Partnerships. James is currently the Director of IT at United Learning, a Multi-Academy Trust of over ninety academies and independent schools, where he leads on their Cloud First strategy and developed the Digital Strategy for the Group. He also led the EdTech Demonstrator programme on behalf of the Department for Education. Patrick is the Head of Education Strategy at the Texthelp Group, an organisation whose mission is to create technologies that can help people read, write and research with confidence, ensuring that everyone reaches their full potential. The session along with panel discussion were facilitated by Charlotte Dougan from the Impact Partnerships team at ImpactEd.

‍Key discussion points

The session was attended by over 40 participants who engaged in active discussion on evaluation and edtech. The key points of discussion were:

• It can be challenging to evaluate whether EdTech is having a direct impact on pupils' and students’ learning or whether EdTech is facilitating pedagogical changes which in turn have an impact on pupils’ and students’ learning.

• A good example of how to untangle this comes from our paper, showing that Oak users reported that by using Oak they save time on planning and resourcing lessons, and that then gave them more time to identify and support more vulnerable pupils (page 6, Evaluating EdTech)

• Across the board, panellists recognised that a key step in  delivering a successful evaluation is establishing clear outcomes and using appropriate tools for measuring them.. 

• ImpactEd understands that this step can seem complicated and without a definite starting point to organisations who are at the beginning of their evaluation journey. Our paper offers guidance on what a good starting point could be for this crucial step. (page 11, Evaluating EdTech)

‍What’s next?

The Talking Impact event is a termly one and the next iteration will happen during the Autumn Term of 2023. . Keep an eye out on our socials for what our next theme will be! If you have any suggestions of topics you’d love to see captured, do feel free to reach out to us on hello@impacted.org.uk