Schools, skills, and learning: The impact of COVID-19 on education

The COVID-19 pandemic is first and foremost a health crisis. Many countries have (rightly) decided to close schools, colleges and universities. The crisis crystallizes the dilemma policymakers are facing between closing schools (reducing contact and saving lives) and keeping them open (allowing workers to work and maintaining the economy).
The severe short-term disruption is felt by many families around the world: home schooling is not only a massive shock to parents’ productivity, but also to children’s social life and learning. Teaching is moving online, on an untested and unprecedented scale.
Student assessments are also moving online, with a lot of trial and error and uncertainty for everyone. Many assessments have simply been cancelled. Importantly, these interruptions will not just be a short-term issue, but can also have long-term consequences for the affected cohorts and are likely to increase inequality.

Impacts on education: Schools

Going to school is the best public policy tool available to raise skills. While school time can be fun and can raise social skills and social awareness, from an economic point of view the primary point of being in school is that it increases a child’s ability. Even a relatively short time in school does this; even a relatively short period of missed school will have consequences for skill growth. But can we estimate how much the COVID-19 interruption will affect learning? Not very precisely, as we are in a new world; but we can use other studies to get an order of magnitude.
Two pieces of evidence are useful. Carlsson et al. (2015) consider a situation in which young men in Sweden have differing number of days to prepare for important tests. These differences are conditionally random allowing the authors to estimate a causal effect of schooling on skills. The authors show that even just ten days of extra schooling significantly raises scores on tests of the use of knowledge (‘crystallized intelligence’) by 1% of a standard deviation. As an extremely rough measure of the impact of the current school closures, if we were to simply extrapolate those numbers, twelve weeks less schooling (i.e. 60 school days) implies a loss of 6% of a standard deviation, which is non-trivial. They do not find a significant impact on problem-solving skills (an example of ‘fluid intelligence’).
A different way into this question comes from Lavy (2015), who estimates the impact on learning of differences in instructional time across countries. Perhaps surprisingly, there are very substantial differences between countries in hours of teaching. For example, Lavy shows that total weekly hours of instruction in mathematics, language and science is 55% higher in Denmark than in Austria. These differences matter, causing significant differences in test score outcomes: one more hour per week over the school year in the main subjects increases test scores by around 6% of a standard deviation.
In our case, the loss of perhaps 3-4 hours per week teaching in maths for 12 weeks may be similar in magnitude to the loss of an hour per week for 30 weeks. So, rather bizarrely and surely coincidentally, we end up with an estimated loss of around 6% of a standard deviation again. Leaving the close similarity aside, these studies possibly suggest a likely effect no greater than 10% of a standard deviation but definitely above zero.

Impacts on education: Families

Perhaps to the disappointment of some, children have not generally been sent home to play. The idea is that they continue their education at home, in the hope of not missing out too much.
Families are central to education and are widely agreed to provide major inputs into a child’s learning, as described by Bjorklund and Salvanes (2011). The current global-scale expansion in home schooling might at first thought be seen quite positively, as likely to be effective. But typically, this role is seen as a complement to the input from school. Parents supplement a child’s maths learning by practising counting or highlighting simple maths problems in everyday life; or they illuminate history lessons with trips to important monuments or museums. Being the prime driver of learning, even in conjunction with online materials, is a different question; and while many parents round the world do successfully school their children at home, this seems unlikely to generalise over the whole population.
So while global home schooling will surely produce some inspirational moments, some angry moments, some fun moments and some frustrated moments, it seems very unlikely that it will on average replace the learning lost from school. But the bigger point is this: there will likely be substantial disparities between families in the extent to which they can help their children learn. Key differences include (Oreopoulos et al. 2006) the amount of time available to devote to teaching, the non-cognitive skills of the parents, resources (for example, not everyone will have the kit to access the best online material), and also the amount of knowledge – it’s hard to help your child learn something that you may not understand yourself. Consequently, this episode will lead to an increase in the inequality of human capital growth for the affected cohorts.

Assessments

The closure of schools, colleges and universities not only interrupts the teaching for students around the world; the closure also coincides with a key assessment period and many exams have been postponed or cancelled.
Internal assessments are perhaps thought to be less important and many have been simply cancelled. But their point is to give information about the child’s progress for families and teachers. The loss of this information delays the recognition of both high potential and learning difficulties and can have harmful long-term consequences for the child. Andersen and Nielsen (2019) look at the consequence of a major IT crash in the testing system in Denmark. As a result of this, some children could not take the test.  The authors find that participating in the test increased the score in a reading test two years later by 9% of a standard deviation , with similar effects in mathematics. These effects are largest for children from disadvantaged backgrounds.

Importantly, the lockdown of institutions not only affects internal assessments. In the UK, for example, all exams for the main public qualifications – GCSEs and A levels – have been cancelled for the entire cohort. Depending on the duration of the lockdown, we will likely observe similar actions around the world. One potential alternative for the cancelled assessments is to use ‘predicted grades’, but Murphy and Wyness (2020) show that these are often inaccurate, and that among high achieving students, the predicted grades for those from disadvantaged backgrounds are lower than those from more advantaged backgrounds. Another solution is to replace blind exams with teacher assessments. Evidence from various settings show systematic deviations between unblind and blind examinations, where the direction of the bias typically depends on whether the child belongs to a group that usually performs well (Burgess and Greaves 2013, Rangvid 2015). For example, if girls usually perform better in a subject, an unblind evaluation of a boy’s performance is likely to be downward biased. Because such assessments are used as a key qualification to enter higher education, the move to unblind subjective assessments can have potential long-term consequences for the equality of opportunity.
It is also possible that some students’ careers might benefit from the interruptions. For example, in Norway it has been decided that all 10th grade students will be awarded a high-school degree. And Maurin and McNally (2008) show that the 1968 abandoning of the normal examination procedures in France (following the student riots) led to positive long-term labour market consequences for the affected cohort.
In higher education many universities and colleges are replacing traditional exams with online assessment tools. This is a new area for both teachers and students, and assessments will likely have larger measurement error than usual. Research shows that employers use educational credentials such as degree classifications and grade point averages to sort applicants (Piopiunik et al. 2020). The increase in the noise of the applicants’ signals will therefore potentially reduce the matching efficiency for new graduates on the labour market, who might experience slower earnings growth and higher job separation rates. This is costly both to the individual and also to society as a whole (Fredriksson et al. 2018).

Graduates

The careers of this year’s university graduates may be severely affected by the COVID-19 pandemic. They have experienced major teaching interruptions in the final part of their studies, they are experiencing major interruptions in their assessments, and finally they are likely to graduate at the beginning of a major global recession. Evidence suggests that poor market conditions at labour market entry cause workers to accept lower paid jobs, and that this has permanent effects for the careers of some. Oreopoulos et al. (2012) show that graduates from programmes with high predicted earnings can compensate for their poor starting point through both within- and across-firm earnings gains, but graduates from other programmes have been found to experience permanent earnings losses from graduating in a recession.

Solutions?

The global lockdown of education institutions is going to cause major (and likely unequal) interruption in students’ learning; disruptions in internal assessments; and the cancellation of public assessments for qualifications or their replacement by an inferior alternative.
What can be done to mitigate these negative impacts? Schools need resources to rebuild the loss in learning, once they open again. How these resources are used, and how to target the children who were especially hard hit, is an open question. Given the evidence of the importance of assessments for learning, schools should also consider postponing rather than skipping internal assessments. For new graduates, policies should support their entry to the labour market to avoid longer unemployment periods.
SOURCE: voxeu.org

References

Andersen, S C, and H S Nielsen (2019), “Learning from Performance Information”, Journal of Public Administration Research and Theory.
Bjorklund, A and K Salvanes (2011), “Education and Family Background: Mechanisms and Policies”, in E Hanushek, S Machin and L Woessmann (eds), Handbook of the Economics of Education, Vol. 3.
Burgess, S and E Greaves (2013), “Test Scores, Subjective Assessment, and Stereotyping of Ethnic Minorities”, Journal of Labor Economics 31(3): 535–576.
Carlsson, M, G B Dahl, B Öckert and D Rooth (2015), “The Effect of Schooling on Cognitive Skills”, Review of Economics and Statistics 97(3): 533–547
Fredriksson, P, L Hensvik, and O Nordström Skans (2018), “Mismatch of Talent: Evidence on Match Quality, Entry Wages, and Job Mobility”, American Economic Review 108(11): 3303-38.
Lavy, V (2015), “Do Differences in Schools’ Instruction Time Explain International Achievement Gaps? Evidence from Developed and Developing Countries”, Economic Journal 125.
Maurin, E and S McNally (2008), “Vive la revolution! Long-term educational returns of 1968 to the angry students”, Journal of Labor Economics 26(1): 1-33.
Murphy, R and G Wyness (2020), “Minority Report: the impact of predicted grades on university admissions of disadvantaged groups”, CEPEO Working Paper Series No 20-07 Centre for Education Policy and Equalising Opportunitites, UCL Institute of Education.
Oreopoulos, P, M Page and A Stevens (2006), “Does human capital transfer from parent to child? The intergenerational effects of compulsory schooling”, Journal of Labor Economics 24(4): 729–760.
Oreopoulos, P, T von Wachter, and A Heisz (2012), “The Short- and Long-Term Career Effects of Graduating in a Recession”, American Economic Journal: Applied Economics 4(1): 1-29.
Piopiunik, M, G Schwerdt, L Simon and L Woessman (2020), “Skills, signals, and employability: An experimental investigation”, European Economic Review 123: 103374.
Rangvid, B S (2015), “Systematic differences across evaluation schemes and educational choice”, Economics of Education Review 48: 41-55.