In this episode, Phil Hill, Jeanette Wiseman, and Kevin Kelly discuss recent studies looking at student learning in K-12 during the pandemic.
Podcast: Play in new window | Download
Subscribe: Apple Podcasts | Google Podcasts | RSS | More
Hosts:
- Phil Hill
- Jeanette Wiseman
- Kevin Kelly
Links:
Transcription:
Phil: Welcome to COVID Transitions, the podcast where we discuss the transition that higher education and even K-12 is going through due to the pandemic this year. I’m Phil Hill and again, I’m here with Kevin Kelly and Jeanette Wiseman. It’s great to talk to you guys yet again. Another somewhat difficult subject as we look at K-12 learning loss.
How are you guys doing? Jeanette?
Jeanette: Doing OK. You know, hanging in there.
Phil: You got the kids partially quiet for the podcast.
Jeanette: We’ll see.
Phil: Kevin, you’re all set up now that we got the repairman gone?
Kevin: He has done his work. He’s done it well. I’m ready to go.
Phil: Ok, great. Well, the topic we wanted to cover today is that we’re starting to see a lot of data roll in, particularly the K-12 level, looking at learning itself. How much are students not learning compared to what they would [00:01:00] have been expected to learn in a normal circumstance? Some people call this a learning loss or COVID slide. It’s basically saying how are students performing from a learning perspective compared to, say, last year or what we would have expected to do?
Initially we talked about doing this podcast because The Washington Post reported on data from the Fairfax County public schools, talking how across the board for middle school and high school, the number of F’s that students were getting for the first quarter were going up significantly. Then we also had some initial data from Illuminate, which they provide some assessment and data platforms as a vendor. They were looking at some of the data and what is the learning loss relative to the annual monthly growth they would have expected. Both of these [00:02:00] reported pretty significant learning losses, and that’s how we wanted to cover the podcast.
Since that time, we’ve had new data come in. In particular NWEA, which is another assessment company, they put out a report looking at 4.4 million students. Unfortunately, there is a pretty big caveat in what they presented. What was reported is the fact of yes, we’re seeing significant learning losses for math, particularly in the middle grades, and that we certainly have differences with different ethnic groups or English language learners. It also got reported that reading really hasn’t suffered significantly. I found that was a pretty flawed analysis that you can’t ignore or just tack on the caveat, but really called into question how serious is the learning loss?
We’ll put these in the show notes, by the way. [00:03:00]We also have a new report from Renaissance Learning Software Vendor that includes some assessment where they’re looking at learning, and they have a pretty comprehensive report. They’re showing significant losses as well. Then McKinsey, the management consulting firm, put out a report that largely relies on NWEA data.
The point is we’re getting a lot of new data looking at the learning loss. It certainly seems to be real, although there’s a question, particularly in reading about how to interpret the data or how significant the data is. That’s what we wanted to cover today, to sort of look at what are we finding out from these early studies. Then later in the podcast episode, we’ll discuss what are the implications for higher education.
Start with you, Kevin. You’ve done some of this research. I guess part of the question is why are we getting this for the K-12 field before higher education? [00:04:00] What are your initial impressions?
Kevin: Well, I think part of it is that higher education looks at retention and success after the term is over, whereas NWEA, Renaissance and other groups that conduct these assessments do them in the middle of the term so they’re able to get data and analyze it earlier than higher ed can.
When we look at some of the question marks that you’ve raised, I think Renaissance did a better job comparing fall 2019 to fall 2020. The same students, even though as a pool of five million, it was considered a small pool, but it was a better comparison, didn’t leave people out. Some of the things that we need to look at, though, are the same things we need to look at in higher ed, where the averages that we’re getting in these reports don’t show what specific demographics are experiencing. I liked how The McKinsey [00:05:00] report pulled out specific demographics for black, Hispanic and lower socioeconomic scale students and the impact that they’re experiencing is greater than for others.
Phil: Yeah, and probably should be a little bit more clear. The caveat that I’m talking about is in the NWEA, they do these assessments already. The previous year they had 5.2 million students taking the test. This year they had 4.4 million. Predominantly the people not taking the test this year come from schools that serve much more of a lower socioeconomic background, the exact students who are most at risk. That’s the caveat they were referring to.
If you look at the studies, what we’re seeing is, yes, those are the students, and I would add in English language learners, that are suffering the most. It’s beyond just a caveat. [00:06:00] It actually questions the analysis. I agree with you, Kevin, I think the Renaissance did a much better treatment of the overall subject. I think even Illuminate did as well.
Jeanette, how does this compare to what you’ve seen, including your own personal experience?
Jeanette: Oh, gosh. Well, I try to be sort of a glass half full type of gal.
These reports and those studies are just really depressing and bleak to me, I think primarily because we’re seeing the commitment and the investment that needs to be made in education 10 years ago didn’t happen. The gaps were there. The chance for such a drop, and especially Hispanic, black and lower income students is going to be great unless there’s a big commitment moving forward both in investment and quality of instruction, no matter if they’re online or in class.
I see it with [00:07:00] my own kids. I know that we come from a place where education obviously is really important. My kids are getting the support. They need it. I don’t think they’re getting it from school. I have the luxury of being able to hire outside tutors to help them in those areas. I’m also at home. I’m not a parent that has to leave the house for work. I’m here to support them as well.
These numbers are showing that unless as a collective community, see how important these slides are into our overall economy and growth for the United States as a whole and globally, because this is happening on a global scale, then we could be in big trouble in five or ten years from now.
That’s frightening to me.
Phil: Some of these studies are pointing out this is not just a temporary blip. This is expected to have a long term impact.
Jeanette: Right. [00:08:00] I think that’s the thing that is just really scary.
The number of dropouts we could potentially see in high school. I think there’s another thing that’s exasperating all of this. My daughter, my two daughters are in high school. One’s a freshman, one’s a senior. They haven’t been in school since March. There has not been one in-person class in Albuquerque, and that means they’ve missed out. Although my freshman daughter made the volleyball team, she went to two practices and they had to cut those down. There’s been no extracurriculars, there’s no orchestra, presentations or competitions. There’s nothing that they normally have. It’s been a tough road.
I know for most parents, just on the mental side of things, as well as trying to keep spirits up for these K-12 students and as it is for everybody during this time. The impacts that it could be having, especially [00:09:00] for some of these students that aren’t getting the additional support and have other pressures, I just feel like the equity issues that we’re going to see for maybe a lifetime could be really dramatic.
Very much a glass half empty kind of view. It’s scaring me a little bit, to be honest.
Phil: Let me take a stab at the common themes that we’re seeing and then just get you guys to react to it.
Even though there are different ways of doing assessments, all the way from looking at grades to actually formalize assessments from different companies is that we’re definitely seeing learning loss. It’s worse for math than it is for reading, although I would argue both exist. Math seems to be suffering more than reading, which is a little bit interesting because it seems like there’s more of an opportunity for some of the courseware and automated tool usage with remote. The [00:10:00] data are showing that it’s actually worse for math, it tends to be worse in the middle grades in terms of grades four through eight or nine than the learning loss that’s happening at one through three or in high school. We’ve already mentioned that the learning loss has a greater impact among disadvantaged student groups, whether it’s black or Hispanic, lower income, English as a second language. Those are the groups that tend to be suffering more than others. It’s not quite clear if the gap is getting worse. However, if you look at Renaissance learning, they actually have showed that, yes, while it’s not getting worse in terms of people who were performing academically worse before the pandemic. Socioeconomic, that gap might be growing. In terms of who was already doing well or not, there’s sort of common learning losses.
Those are some of the themes I guess I would [00:11:00] see. What would you guys correct or add in to sort of common descriptions?
Kevin: I would add in another at risk group as students with disabilities because there’s still not universally designed approach to this emergency remote moving towards online educational experience. Then there was an article in Educational Researcher, one of the authors, I think was an NWEA person, but they looked at projections based on absenteeism, literature, and summer learning patterns to see if they could determine potential pitfalls.
They thought that losing ground during school closures wouldn’t be universal. The top third of students may actually make some gains in fields like reading. It makes me again, think back to when you average everything out. It looks like things aren’t changing that much, but when you look at it by subgroup, then [00:12:00] you find those bigger discrepancies.
Then to the point that Jeanette was making earlier about the long term effects downstream, the McKinsey report had a couple of things that stood out to me. One, the loss of learning leads to loss of earning. They had an exhibit that showed that the economy right now would be significantly larger if it had closed achievement gaps in education ten years ago. Those are the types of things that we have to see what is going to be the impact of this pandemic, not just in the next year. McKinsey said the effects are not likely to be temporary shocks easily erased in the next academic year. We have to look at this as something we should be keeping an eye on for the next three to five years or maybe more.
Jeanette: This is me being an optimist again. I think some of those achievement gaps could be closed, but the investment needs to be made in education in terms of [00:13:00] even professional development with the wonderful teachers right now making sure that they’re doing it. That hasn’t been there. That’s why a lot of those achievement gaps weren’t closed 10 years ago. I feel like it needs to be made again. That’s the way that we can close it. I think it’s unsane if we’re going to be doing that, especially because overall revenues and budgets are down everywhere and people just don’t have the funding to make some changes that were needed prior to COVID.
Phil: That’s a good point. Let me take it up a level, meta level, but also somewhat depressing. If you look at it this way, there’s been a persistent talk that emergency remote teaching is not the same thing as well-designed online learning and that it’s unfair to make conclusions about online learning based on a lot of what’s happening today. Take that as a given. However, I think that what we can say in a common theme is if you look [00:14:00] at K through 12, what we’re doing this year is not working.
What we’re doing, particularly in terms of not being face to face, is not working in terms of learning, and it’s disproportionately affecting students who can’t afford to to lose anymore. Let me just state that and see if any comments on that, if you agree with that perspective.
Kevin: I would say it’s not working because they haven’t really thought through the online experience. I was contacted based on one of my blog posts by someone here in California who was trying to convince the board to allow parents to have their students have Synchronoss online options if they didn’t want to risk having their child in school, if there are people at high risk categories at home that they don’t want to get COVID. The board basically said it’s either face to face in the school, or you [00:15:00] can do this teach yourself online program that basically as this meets the same learning outcomes, but there’s no teacher interaction, you just complete assignments, which requires heavy investment of time from parents or someone in the home.
That, to me, is not a well-designed instructional experience for every circumstance, trying to create some flexibility for people who work and other things. I would say you’re right that we need to be thinking about whether or not teaching and learning is moving toward a well-designed online experience. I don’t think we’ve gotten there yet. That partly speaks back to the investment that Jeanette’s been bringing up all along.
Phil: I guess part of what I’m pushing is saying, yes, that’s all true, but we first need to look at the students we have today and say we need to recognize there’s a serious [00:16:00] downside to what we’re doing today through K-12. We’re seeing some of the impact of it. It’s not just a matter of what could happen. It’s a matter of acknowledging what is happening.
Jeanette: I think the government officials have been in tough positions of trying to make really smart and critical public health decisions. I think in the spring, at least, I felt like they were somewhat of an emergency situation. Right now, though, I’ve had friends that are teaching elementary school and the ones that are in the lower grades. Those kids are just struggling.
I look at my kids, middle school and high school, and they’re able to be somewhat self directed. They’re fairly motivated to get things done. These kids that are especially K through third grade, there’s an entire learning curve of how to use the computer. [00:17:00] They still don’t even know how to read. If they’re in a house where there’s either parents that English isn’t their first language and there’s not that support there or parents that are trying to work, or maybe they’re there with older siblings, that learning is not happening and they see it. There’s only so much they can do as teachers to try to reach those younger students. I think it’s a huge issue.
I think what you’re trying to say, Phil, is maybe these public officials need to start recognizing just the overall impact on the mental health of these students, the equity issues that are coming up that they’re not going to be able to overcome through technology or online learning, and that there’s got to be a balance between these public health decisions and what needs to be happening in the schools. It’s a hard balance.
Phil: I mean, if you want to stated eloquently like you just did. Yes, that’s [00:18:00] exactly what I meant.
Let’s turn this to higher education. What are the impacts or implications for higher education from what we’re seeing with some of this early data in the K-12 space? I guess I’ll just throw out there. I think there are two that at least seem obvious to me.
One is these are the higher education students of the future or potential students. We’re talking about long lasting learning impact on what they’re learning, particularly in math. Then there’s also the thing of we just don’t quite have as much data in higher education, but what are we expecting to see and what today’s college students are learning or not learning through this pandemic? I think that this is very tightly tied to higher education. I’d be curious to hear what you guys think, these implications, how is this going to hit higher education?
Kevin: Or how is it already hitting? [00:19:00]
Phil: Yes.
Kevin: We know from the TopHat survey that went out in October, almost 70 percent of the students said they did not feel they learned as well online. Whether or not we have the results of their learning, we know that they don’t feel they learn well. That’s backed up by the article in Inside Higher Ed this Monday stating that fall students are seeking the same pass fail options that they had in spring, even though colleges seem less willing to do that this time around.
BestColleges.com had an article back in October talking about digital divide, talking about things like housing and security as a factor. Now that students are the campuses are closed, they may not have anywhere to go if they don’t have a stable home situation that they’ve left. That article in particular brings up that same issue that Jeanette’s been bringing up. The lack [00:20:00] of investment in distance education for some colleges and universities, pre COVID is causing a challenge now. It’s something that Phil has brought up in some of our previous COVID Transition podcasts, where he says those that have taken the time and spent the money to work on these things beforehand are going to be the ones that are way farther ahead now. I think we’re just seeing that amplified as we go through the fall semester.
Phil: Jeanette, what are you thinking the implications are going to be?
Jeanette: Well, if you look at the McKinsey report that came out, they’re estimating close to a million new high school dropouts because of COVID. I’m actually exaggerating because I think when you go back, it’s 648 thousand.
Phil: Oh, you rounded up.
Jeanette: I mean, that’s a lot. That’s just what they’re estimating. I mean, I think that would be great if it ended up being a lower number. Right there, that is going to be an impact [00:21:00] in community colleges, maybe students that had considered going to some kind of post high school education, they’re not going to necessarily go that way. How is that going to impact local communities that were relying on that revenue. Not only the revenue, but also that educated workforce to help their economies in the coming years? Right there, I think that’s a huge impact that’s going to happen.
I think also just in terms of learning loss. For years we’ve heard that faculty have been saying that many of their freshman college students come into college, not prepared to be a college student, that they don’t have the skill set, whatever, in math or English and writing and all of those things. I just don’t see how those college freshmen going in who have experienced this learning loss because of COVID starting their freshman year in the same place that maybe three or four years ago where there [00:22:00] are already those complaints. It seems like that’s going to be a lot of work for faculty to get people back up to speed. Those are two areas that I think they’re likely to happen.
Kevin: To piggyback on one point that Jeanette made about the the dropout rate and the potential impact for students entering higher ed, we already saw that in a previous episode. We talked about the decrease in enrollment, especially in the community college level. What was it? Twenty three percent drop in first year student enrollment.
Phil: I think subsequent data knock that down to 19 percent.
Kevin: Ok, still one in five. Yeah, that’s where people are leaving the educational process. Whether or not they’re inspired to or encouraged by outsiders to come back is going to be the trigger. Or if they decide that they’re going to have to just enter the workforce and see what happens.
Phil: Yep. Well, I’ll steal something from Kevin who found this National Bureau of Economic Research working paper. In this working paper, they [00:23:00] looked at seven economics courses taught at four R-1 institutions in the U.S., research universities. I’ll just quote from the abstract. “We find that students perform substantially worse on average in spring 2020. Looking at the spring term,” that was sort of cut in half when compared to spring or fall 2019, “we find no evidence that the effect was driven by specific demographic groups. However, our results suggest that teaching methods that encourage active engagement, such as the use of small group activities and projects, played an important role in mitigating this negative effect.”
In this case, don’t want to extrapolate it too much, it’s early working paper. It’s in economics research university. At least in that context, it’s interesting that their initial findings are [00:24:00] in higher education, not correlating as much to the demographic groupings, but it also points to the well-designed aspect. The results suggesting that teaching methods, active engagement, small group activities and projects really mitigated the learning loss in higher education. It would be interesting to sort of see a broader view here. That certainly, I think, points out the same things we’ve been talking about. We are seeing significant learning impacts and we will continue to have to deal with them. However, an important element of that is that you get less of an impact and it’s mitigated. We’re even seeing some gains here and there based on people who have really well designed their courses, particularly around active engagement.
To end on a positive note, I certainly hope this type of research that we see more of it because at least [00:25:00] we have a chance to make some improvements based on how we do the teaching. I don’t know if that’s too positive, but that’s one way I look at that data.
Kevin: I think it’s positive in that the potential is there, but it means that you have to continue to move mountains. I think I’ve referenced in previous podcast, Cal State L.A. served over a thousand instructors with their workshops this past summer. That’s getting people ready. It’s not giving them life experience and online teaching for years and years and knowing how to facilitate things. They know how to set them up.
That’s going to take another semester or two for these instructors to really adopt these principles and strategies and stop recreating a face to face class where it’s talking head on Zoom and then go take a quiz on the learning management system. We need these types of engagement and do it and do it well, requires some trial and error periods. The students are suffering through that right now.
Phil: To [00:26:00] tie it with what Jeanette said, we know one of, if not the main area, that where we need this investment, it’s in professional development for faculty and support. We know what we need to do more of, but it’s going to take some investment during a very difficult time. We do know some of the key things that will mitigate the pain we’re going through.
It’s important to understand the aspect of what’s happening. I think this is important data coming out of K-12. It’s good to take a look at it, but we should definitely follow up and start looking for end of term learning reports from higher education, because I think that will be important to understand as well. Even if your only focus on higher ed, this K-12 data has some implications that you need to be aware of.
It’s great talking to you. To [00:27:00] the eloquent Jeanette, who really did a great job. I really do mean that. Kevin, thanks for finding a lot of this research as well.
Kevin: You bet.
Phil: I hope you enjoy your weekend and next week. Thanks again. Bye.
