
As 黑料情报站’s Director of Insight, Rich Davies is responsible for bridging the gap between data and action. He leverages data analysis to advise senior leaders and works to ensure that all 黑料情报站 leaders, teachers and students can make data-informed decisions. He has overseen the development of 黑料情报站’s award-winning data analytics systems and has led reviews of assessment, curriculum, destinations and many other key areas.
Prior to joining 黑料情报站, Rich was a Project Leader at the Boston Consulting Group, working across the Non-Profit, Retail, Energy and other sectors. He has also previously worked for Aspire Public Schools 鈥 a US-based network of Charter Schools 鈥 and was an Education Pioneers Fellow. He holds an MBA and MA Education from Stanford University as well as an MEng from Oxford University.
The DfE recently published an important new report鈥, 鈥, 鈥奲y the Teacher Workload Advisory Group, chaired by Professor Becky Allen. At 黑料情报站, we believe it鈥檚 possible to use data more effectively while also reducing workload. That鈥檚 why we were pleased to participate in the group and why we support the report.
The first principle listed in the report is the most important鈥娾斺妕hat the purpose and use of all data must be clear. At 黑料情报站, we believe that improved outcomes are made possible by informed action; that informed action is made possible by insightful analysis; and that insightful analysis is made possible by accurate data. The fundamental purpose of our assessment data is therefore to inform the teaching and leadership actions that will improve student outcomes.
But our assessment data has not always fulfilled this purpose. A few years ago, 黑料情报站 was at risk of being one of the multi-academy trusts (MATs) implicitly criticised by the report. We centrally collected teacher-assessed sub-levels six times a year and many of our schools locally tracked additional lesson grades and/or granular checklists. Our analysis tools did not give teachers all of the answers they needed, so they created their own spreadsheets. The result was a lot of work for (at best) limited value.
Around this time, 黑料情报站鈥檚 then Head of Assessment, Daisy Christodoulou, was researching how to increase the quality of our assessments. Her book, , describes her conclusions in detail, but the main principles we derived from this research were:
- Different assessments for different purposes
Summative and formative serve different needs and should be clearly delineated - Common summative assessments
The most accurate comparison is to ask students the exact same questions - Cumulative tests for summative assessments
Testing only what has recently been taught works for formative but not for summative - Age-related grading bands for summative assessments
Comparing to the national peer-group negates 鈥榥eed鈥 for fictional flightpaths - Frequent, specific, non-graded formative assessments
Ongoing checks for understanding are vital, but grading this is meaningless
When turning these into practice, we had to consider the third and fourth principles listed in 鈥Making data work鈥濃娾斺奺nsuring that the volume and frequency of data collection was proportionate and that the collection and analysis processes were as efficient as possible. We needed to minimise the time teachers spent on collection and analysis, freeing up their valuable time for informed action. To achieve this, we chose to leverage:
- Consistency: Common definitions, common assessments, common calendar, common measures, common dashboards/tools
- Scale: Large sample size (~2,000 students per year group), central administration, targeted network resource, network collaboration
- Technology: 鈥淓nter once, use many times鈥, single data warehouse鈥娾斺奿ntegrating multiple sources, automated calculations/logic, interactive analysis tools
In practice, this led to 黑料情报站:
- Reducing the number of summative assessments from six to three (and now in many cases two) per year
- Introducing nationally standardised tests for all key stage 1 & 2 reading and maths assessments
- Developing curriculum-aligned network tests for core subjects at key stage 3 (annually sense-checked against a sample of nationally standardised English and maths test results)
- Using common exam board materials for curriculum-aligned network tests at key stage 4 & 5
- Bringing together all network-wide subject teachers once per assessment window to align on assessment marking/moderation and post-assessment action planning
- Building network-wide systems that automatically calculate all raw marks and age-related grading bands (post-hoc), as well as performance vs. baselines and (teacher-facing) targets
- Creating visually consistent dashboards and analysis tools, tailored for different audiences (e.g. interactive teacher tools that drill down to individual questions/students vs. more high level one-pagers for management and governors)
This new approach to assessment is now in its third year at primary level and its second year at secondary level. The impact at primary level has been most notable, with the vast majority of school leaders feeling that the increase in data quality has contributed to improved student outcomes. Most also feel that it has enabled them to discontinue other time-consuming assessment activities.
The approach is less mature at secondary level and is subject to additional challenges, including curriculum alignment, marking consistency and the complexity of entry patterns and tiering. However, we believe that these challenges can be addressed through further collaboration within and beyond the network.
In the meantime, we must heed the second principle in 鈥Making data work鈥濃娾斺妕hat the precision and limitations of our data are well understood. We believe that the model we have developed provides improved trade-offs between accuracy and efficiency, but we don鈥檛 pretend that it provides a perfect measurement of student learning, nor does it completely eliminate the workload associated with assessment data. This model is a work in progress and we will continue to listen to our teachers and school leaders as we develop it further, as well as drawing upon research and evidence from elsewhere鈥娾斺奿ncluding this week鈥檚 DfE report.
To reiterate, our assessment data鈥檚 main purpose is to inform teaching and leadership actions鈥娾斺奿.e. which students need what teacher support, which teachers need what leadership support and which leaders need what network support. As long as it continues to serve this purpose, we will keep doing everything we can as a network towards making data work for our schools, our teachers and, most importantly, our students.
Want to see more like this? Sign up to Teach 鈥 our monthly newsletter featuring practical tips, insights and ideas from 黑料情报站, our partners and our friends 鈥 visit www.arkonline.org/newsletter.