Earlier this month Phil asked a question on Twitter about the growing usage of (and pushback against) faculty training based on the Quality Matters Course Design Rubric. That question led to a rich discussion – both pro and con – on the usage of the QM rubric in the attempt to improve online teaching in Fall 2020. The QM staff requested that we help with an alternate forum for them to address some of the issues raised online.

This is the first in a special series of podcast episodes on an important topic as we try to migrate from emergency remote teaching to purposely-designed quality online education.

  • 15A: Introduction of topic
  • 15B: Interview with Bethany Simunich and Brendy Boyd from Quality Matters
  • 15C: Interviews with Stephanie Moore and Jesse Stommel


Phil: Welcome to COVID Transitions, where we discuss a lot of the transition that higher education has gone through and is going through due to the Covid-19 pandemic. I’m Phil Hill, and I’m here with Jeanette Wiseman and Kevin Kelly. Earlier this month, we had an interesting situation where I put out what I thought was an innocuous tweet asking about why am I starting to see more pushback on Quality Matters and its usage during professional development this summer. I’m not arguing for or against it, but is there something that happened on why this is becoming more discussed out in the open?

For those who don’t know, Quality Matters is a non-profit organization that provides a rubric of course design standards and creates a replicable peer review process, the goals being: training and empowering faculty to evaluate courses against these standards; providing guidance for improving the quality of courses; and certifying the quality of online and blended [00:01:00] college courses across institutions. And boy, it seemed like the Twitter conversation tapped a vein. We got all kinds of conversations going back and forth, a lot of it quite emotional where you get the sense that there was really pent up feeling behind this issue, that this is a topic that’s really hitting people right now and then they’re starting to let it out. Twitter is not the best medium to explore topics in more depth, and we agreed with Quality Matters request to provide a different forum.

Hence the special podcast series. In this first episode, Jeanette, Kevin and I introduce the topic. In the second episode, I interviewed Bethany Simunich and Brenda Boyd from Quality Matters to hear their perspective directly and in depth. And the third episode, I interviewed Stephanie Moore from the University of New Mexico and Jessi Stommel from Hybrid Pedagogy as they provide a critical perspective, albeit with constructive criticism and suggestions. [00:02:00]

But before we do that, the first thing that struck me surprised me all of the responses we got, but I guess the general sense that I got on why this became a big topic is because of how Quality Matters is getting implemented, particularly now as a method for administrators and schools to try to get either control over online education, or to help them migrate what they think is moving from emergency remote teaching to true online education. So it’s becoming the tool to say we have to get all faculty doing quality online education. And it’s the way that it gets applied is a huge portion of why there’s a lot of frustration and emotion out there.

But to get started, did this discussion surprise both of you guys? And, you know, why do you think there was such a strong online discussion on this topic? [00:03:00]

Kevin: I think there’s a couple of things at play here. Right. So you pointed out that this is in response to Covid, campuses are implementing processes at higher rates of speed. So you get this combination of forces, right? It’s like the wedge in Southern California, Newport Beach, where two different vectors of waves form this massive wave. That’s really fun to ride, but really scary. And people can get hurt. You have the use of a rubric which is in and of itself, not the the challenge. And when you say innocuous tweet, I don’t know if those exist anymore.

Phil: I got accused of that, by the way.

Kevin: Yeah, but I think, you know, when Stephanie Moore brought up this being used as a cudgel to force us all to mean regression, then she’s pointing out that, hey, some people have already been working on their online courses and may not need to go through the same process. Other faculty members are being asked to do something very quickly so it can get the [00:04:00] feeling of being like a Play-Dough factory and everything is going to look the same, maybe a different color, but it’s going to be the same shape and dimension. But, you know, I’ll stop there for a second. But there’s so much to say about the difference between using a course review process to improve the experience for learners and then implementing a course review process very quickly to standardize or try to guarantee a sense of quality to, you know, assuage the fears of students and parents and but also just to comment on that.

Phil: But there’s been building thoughts and emotions on this topic over the past few months, it’s becoming apparent to me as well, people have wanted to have this discussion.

Jeanette: I wonder if a lot of this is really based on just overall frustration. To some extent. I think that there’s likely a lot of people that are trying their best to get these courses up and running. I think the ones, people that are experienced [00:05:00] with running online courses and doing instructional design and pedagogy online, and they’re comfortable with that, find these rubrics to be confining and something that doesn’t allow them to really show and teach the way they want to.

And I’m wondering if that’s where I’m seeing a lot of the pushback is those people that are just like, ‘hey, I know what I’m doing. Please don’t make me do this because you’re requiring it.’

Kevin: Well, I’d be interested to explore how many of the tweets are by a part time lecturer or faculty who could really use some guidance, full time tenure track faculty who aren’t used to having their teaching in any form evaluated by a peer, or instructional designers who are used to helping faculty through these challenges. Because I think that those different again, it’s a sense of privilege that tenure track faculty members may enjoy. ‘Hey, my courses, my course, don’t tell me how to teach it.’

And when we talk about, hey, online learners are still [00:06:00] succeeding at lower rates. We just saw all the student surveys telling us how much students weren’t engaged. And the CHLOE survey by chief online officers said the same thing, that the students didn’t feel like they had any form of engagement. It was a very flat experience. So there are ways to use rubrics as a way to make faculty aware of the most common challenges. I know at Peralta we created the equity rubric as a way to make faculty aware of the biases, assumptions and institutional barriers that affect learner motivation and achievement, according to the research. And then we created online training modules to learn about the challenges, analyze what it looks like in a real course, and build their own activities. And so when you see some of these tweets referring to the professional development, the opportunities for conversation, again, I don’t think people are questioning the use of the tools so much as as how it’s being used.

Phil: The people that I know, [00:07:00] and you know have a bias of who you really pay attention to or give credence to if you know who they are, but so much of the pushback against Quality Matters, and where it seemed to tap a vein, if you will, came from people who were experienced online teachers, or instructional designers, but people who are have been already pushing for more faculty to increase their knowledge of how to use online modality to improve teaching and learning. And you got a sense it was more like we know that we need to improve things. And I’m not just speaking for my own personal case, but I’m frustrated that the way Quality Matters is getting implemented is forcing us down a path that I already know is dangerous. And so it’s the over application. I think that there is a pretty good sense that I saw a lot of very thoughtful responses where people were answering not just for themselves, but for what they felt [00:08:00] is needed in the in the market, if you will.

But at the same time, you make an excellent point. A lot of this is how it gets implemented. It’s like, well, somebody said it’s unfortunate that it’s called Quality Matters because it implies course quality comes from the usage of this rubric. So if you want quality online education, as in moving from emergency remote to online education, here’s how you do it. And so you take that sort of mentality, and then you use it as a cudgel to make everybody fall in line. And so I saw a lot of the pushback was argued not on the concept, some was on the concept of the work, but a lot more was on how it gets applied at schools, perhaps overzealously, how a rubric should be viewed. Is it a minimum set of standards? This is a guide to make sure you think of certain aspects, or is it the way to get quality into a course?

Kevin: Well, I’d see [00:09:00] it as a scaffolding device to help online instructors who might be newer to the process begin to improve the course experience based on what the research shows. And so those professional development opportunities that a lot of tweets described in both in response to you and Deb Adair are one place for those conversations to take place. But peer review processes are another where you can have much more in-depth conversations because you’re walking through your course and talking about why you’re doing certain things and you have a chance to hear from a veteran online instructor the way they do things. And so it’s more along the lines of what Jesse Stommel was saying. And others, maybe Peter DeCourcy, who were talking about conversations being part of the process instead of just having these ‘run people through the grist mill.’

Jeanette: I mean, to that point, Kevin, how often do you think that scaffolding is happening? And isn’t that what people are pushing back against?

Kevin: It’s probably the case that because you have to scale up your trainings [00:10:00] over the spring and the summer, that you’re probably going to have less time for conversations, because you’re just putting as many people through some sort of preparation as possible. We saw, and I think it was the CHLOE survey, that people are averaging somewhere between 20 and 40 hours of formal prep for the fall, which is more than a lot of instructors prep for. And then, you know, period, they don’t they don’t get pedagogical training and in this way.

Phil: But I do want to separate, because I think we can explore both. There’s a question not just how could it be used, but how it is being used. And that’s where I saw a lot of the argument saying, no, it’s not being used as scaffolding, it’s being used as the be all and end all. That’s a problem. Now, how can it be used or how should it be used as a separate question?

And to be fair, Deb Adair jumped into the discussion, and she mentioned some [00:11:00] things she had said. ‘It’s not an endpoint. It’s a beginning. It’s always been about being better than good enough. And that’s trying to say, let’s raise the floor for all of it. But that doesn’t mean that talented instructors and instructional designers can’t go further. So it’s ensuring the basic design is in place and helps all students to be successful.’ That was one of the things that she was arguing and the initial feedback.

However, I think there there’s an organizational issue that they need to be careful about as well, because the initial feedback that was happening both privately and within the discussion was, ‘oh, that’s just a bunch of grumbling faculty, that’s the five percent and they’re just grumbling about what they want. They don’t want any standardization, whatever students have.’ But so there was sort of this organizational pushback that they need to be careful about, because most of the comments I saw were not [00:12:00] of that vein of ‘I only care about myself.’ I think it was more people – and you mentioned Jesse Stommel – where they really have thought about what this means broader for education and what’s best for students. So I guess I’m cautioning, or hope they don’t interpret this too much as a bunch of rabble rousers, as opposed to a legitimate discussion that needs to be had and can lead to improvement.

Kevin: Well, I liked how some people point out – it might have been Kelvin Bentley, I’m not sure – but that we need to also be looking at facilitation. It’s something I brought up in that three part series about online course design rubrics on e-Literate that we use these rubrics for the course design process. But very few of the seven major rubrics out there and look at the facilitation process. And that’s actually just as important when you get the students in the online classroom. How are you engaging them and making sure that you’re assessing their learning and authentic ways?

Phil: Rob Gibson, he had jumped in, and I [00:13:00] believe he’s led a lot of Quality Matters training. So, he’s seen how it can be applied based on his direct experience. And he had a lot of good points talking about how the potential of it, that it really can improve teaching and learning. He brought up an issue about accessibility. Here’s a great tool to really force people to deal with accessibility comprehensively through a course. And if you just throw away the baby with the bathwater, where else are you going to get some of this advice to ensure that people aren’t ignoring issues? Kevin, you mentioned the Peralta rubric. Same issue there. If you throw out the baby with the bathwater, do you have another tool that does a better job of making sure that people think of the equity issues involved in education? So I thought that Rob had some very good points about how it can be used. And also almost a caution of if you throw it out, then how are you going to deal with some of these subjects?

Kevin: Right. And [00:14:00] I think it boils down to when groups create these instruments and processes there, they’re done with the right intentions. And so people have to be careful in how they’re applying them, even in cases of emergency. And so I think the conversation is a very good one. And it’s interesting that it took things boiling to a head during a time of crisis for it to emerge into the public speech scene.

Phil: So with that in mind, that’s part of the reason: Let’s take it’s a valuable conversation. Excellent point that you and others have made about Twitter is not the most innocent place to have a conversation. So that’s what we’d like to do. So that so we’re doing a podcast interview to allow two of the thoughtful leaders and the different sides point of view. Let them debate some of these issues in more depth on an important topic and get it out of the Twitter discussion. But looking [00:15:00] forward to hearing from them. And we will definitely like to also discuss what they’re saying, but also the general subject of rubrics, not just about Quality Matters and what the role can be.

Thanks for prepping the field, if you will, Kevin and Jeanette.