New Research: The Impact of the Covid-19 Pandemic on Student Success

A new analysis from Panorama Education’s Data Science and Applied Research team shows the significant impact the Covid-19 pandemic has had on student skills and relationships, including: 

• Clear disruptions to typical student experience during the initial year of the pandemic, followed by recovery toward normal during the following year
• A large and persistent drop in self-efficacy across all grade levels, with most pronounced declines for 9th graders
• A notable decline in some aspects of self-management, particularly in students’ ability to work independently
• Stronger, more positive teacher-student relationships during the first year of the pandemic, particularly for 6th graders

The Panorama Data Science and Applied Research team analyzed survey response data on 11 topics from 217,000 students across 3,000 schools spanning 5 years. 

The Covid-19 Pandemic and Student Success

The Covid-19 pandemic brought unprecedented changes to students’ educational experience—and we’re still learning more about its effects. Shifts between remote, hybrid, and in-person learning, a hiatus on out-of-school activities, and disruptions to interpersonal relationships had significant impacts on:

• Learning loss: A report by the National Association of Education Progress (NAEP) showed that students’ math and literacy scores in 2022 dropped to lows not seen in decades.
• Mental health: Three significant children’s health groups released a joint declaration of a national emergency in child and adolescent mental health in 2021.
• Chronic absenteeism: Pandemic-related learning loss was accompanied by a large increase in chronic absenteeism due to both canceled instructional days and student nonparticipation in instruction. 

Until now, research on pandemic-related impacts has largely been focused on students’ academic outcomes and mental health. Little to no research has been published on how students’ subjective experiences of school and learning has changed, despite being a key component of student success.

To fill that knowledge gap, Panorama’s Data Science and Applied Research team completed a three-year longitudinal study of over 200,000 students’ learning-related experiences during the pandemic. To the best of our knowledge, this is the first study of its kind. In this blog, we’re sharing the results of that study—and how school and district leaders can use the findings as they focus on learning recovery and supporting students’ social and emotional needs.

• Research Methods
• Student Survey Topics
• Research Findings
• Self-Efficacy
• Self-Management
• Teacher-Student Relationships 
• Next Steps for School & District Leaders
  
  

Download Panorama's 360° Survey Starter Pack to get started gathering feedback in your district.

Research Methods

Over the course of the study, 217,000 students from 3,000 schools answered questions from Panorama’s survey instruments. We selected students who stayed in the same school during the three-year study, and looked at students in grades 3, 6, and 9 at the start of the study so that we could investigate how longitudinal changes varied by age. We examined students from two cohorts:

• The pre-pandemic (control) group, assessed in fall of 2017, 2018, and 2019 
• The pandemic group, assessed in fall of 2019, 2020, and 2021

A timeline of the Panorama Data Science and Applied Research team's study, assessing two cohorts of students across 5 years

To control for development or other changes unrelated to the pandemic, both groups responded to the same survey questions each year. By comparing the pre-pandemic and pandemic groups on possible changes in learning skills, educational environments, and relationships, we uncovered significant and ongoing shifts in students’ pandemic learning experience. The table below describes the survey topics included in this study.

Student Survey Topics

As each of these constructs is closely linked to learning in school and success in life, school and district leaders can apply these findings to create stronger systems of support for students to continue to build these skills and relationships.

Research Findings

Overall, we found a few noteworthy trends in students’ experiences. In general, students' educational experience was majorly disrupted during the first two years of the pandemic, with changes over time that were both larger and more variable than typical.

• The changes students experienced over the first two years of the pandemic were much larger than normal. From 2019 to 2020, the changes across all questions were, on average, 47% larger in size than the comparable changes for the pre-pandemic group. And from 2020 to 2021, the changes were 68% larger than normal.
• Longitudinal changes across questions were much more variable for the pandemic group. The changes from the first to second year were over three times as variable for the pandemic group compared to the pre-pandemic group, and the changes from the second to third year were over four times as variable. 

• The pattern of change differed between the groups. We typically see that, for example, if the average student response to a survey question increases one year, it tends to increase the following year. Although this trend is reflected in the pre-pandemic group, we see the opposite in the pandemic students. Responses show that the disruption during the first year of the pandemic tended to be followed by a recovery towards normal in the second year.

Below, we’ve done a deeper dive into three topics that showed significant shifts during the pandemic: self-efficacy, self-management, and teacher-student relationships.

Self-Efficacy

Our research showed that students experienced a drop in self-efficacy during the first year of the pandemic, with recovery during the following year. These effects varied notably by grade level, with younger students showing smaller declines followed by greater recovery. Notably, 9th graders showed a very large drop in self-efficacy during the first year of the pandemic, followed by very little recovery during the second year. This pattern stood in contrast to the slight and constant declines in self-efficacy that 9th graders typically experience over the same time period.

Students who experienced the pandemic reported larger declines in their self-efficacy in the second year of the survey (Fall 2020) compared to students who did not experience the pandemic–and this disparity persisted into the third year (Fall 2021).

These declines could be caused by or contributing to the learning losses observed during this time period. They could also reflect the many other disruptions in students’ educational, social, or family environments. Whatever the cause, it’s clear that students during the first year of the pandemic lost a remarkable degree of confidence in their ability to learn class material and complete schoolwork.

Self-Management

Across all grades, students showed declines in some facets of self-management during the first year of the pandemic. Most notably, students struggled to stay focused when working independently, as shown in the figure below. They also showed a difference in how they responded to the question: “During the past 30 days, how often did you get your work done right away, instead of waiting until the last minute?” In 2019, for example, 65% of students responded favorably to this question, compared to only 57% of students in 2020. This 8% drop was much larger than the comparable 3% drop for the pre-pandemic group.

These declines in certain aspects of self-management, particularly the ability to work independently, may help to explain pandemic-related learning loss, and reflect a hallmark experience of remote learning.

During the second year of the study (Fall 2020), students who experienced the pandemic reported significant declines in being able to stay focused when working independently. However, this appears to have returned to pre-pandemic levels by the third year (Fall 2021).

Teacher-Student Relationships

In sharp contrast to declines in self-efficacy and self-management, students reported relatively stronger, more positive relationships with their teachers during the first year of the pandemic. We saw this across all facets of teacher-student relationships, and particularly large for 6th graders. Moreover, this effect was temporary: teacher-student relationships largely reverted to normal during the second year of the pandemic.

In the years before the pandemic, students typically reported declines in their relationship with their teachers in the second year of our surveys. Students experiencing the pandemic, however, reported a slight increase in their relationship with their teachers in year 2.

For example, teacher-student relationships typically deteriorate as students progress from 6th to 7th grade. We see this in the pre-pandemic group: Over that time period, nearly 1 in 10 students lost confidence that their teacher really cared about how they were doing. For the pandemic group, however, students were actually more likely to believe that their teachers cared about them. 

These findings show that during the first year of the pandemic students felt unusually supported by and close to their teachers, potentially through a combination of increased one-on-one interactions, an emphasis on well-being over academic learning, and the bonding that comes from navigating life-threatening circumstances together.

Sixth graders experiencing the pandemic maintained strong relationships with their teachers into seventh grade compared to the declines seen previously. However, the students experiencing the pandemic dropped to previously seen levels by eighth grade.

Next Steps for School & District Leaders

Across the board, the Covid-19 pandemic led to drastic changes in students’ social, emotional, and academic experiences: in their perception of themselves, in their relationships with others, and in their experiences of school. 

Research shows that the pandemic was—and remains—a major disruptive event in students’ internal lives. We know that our schools and educational systems have not yet returned to normal, and we will continue to learn more about the pandemic’s impact in the years to come.

However, the pandemic showed that educators and students are capable of tremendous flexibility. And in the face of hardship, great caring and compassion helped students and adults demonstrate resilience. For many schools, attention to students' social and emotional well-being was an important part of this process. By prioritizing students’ social and emotional growth, districts can support learning recovery efforts and improve student well-being.

Education leaders can use these research insights to:

• Anchor on relationships, community, and connection in classrooms and school buildings
• Focus on learning recovery and systems and policies that serve students’ social and emotional needs
• Continue to reimagine the ways we support our students and educators for the better.
  
  

Download Panorama’s 360° Survey Starter Pack to elevate and act on student, teacher, and community voice.