an Excelsior University site
A recording of the webinar held on April 5, 2019.
| 0:03 | Francesco: All right. Welcome everyone. This is the fourth and final installment of the Writing in the Disciplines Training Series. |
| 0:09 | This installment will focus on providing effective feedback for students and also on using rubrics which is part of providing effective feedback. |
| 0:20 | This is a project that is brought to you through a collaboration of SUNY colleges and Excelsior University. |
| 0:28 | These training modules—these webinars and seminars—are being delivered at multiple campuses throughout the week. |
| 0:39 | I’d like to introduce myself. I am Francesco Crocco, the director of the Online Writing Lab at Excelsior University, and I will share that I am local in Lafayette, Louisiana. |
| 0:52 | I know as a webinar, that folks here are far flung in different parts of the country, so what I’d like to do is invite you to introduce yourself. Tell us your name, your title, and what you do at Excelsior and also where you physically are located. |
| 1:08 | I’m just going to go down the list here. We’ll start with Amy Erickson. |
| 1:16 | Amy: Hi. I’m Amy Erickson. I’ve been working for Excelsior since 2012. I teach in the English department. I teach humanities. I teach in the business for business writing, and I am physically in Arizona at the moment. |
| 1:32 | Francesco: Oh wow. Where in Arizona? |
| 1:35 | Amy: I don’t know if you’ve ever heard of it. It’s called Surprise. |
| 1:39 | Francesco: I have heard of Surprise, Arizona. I love that name. I’ve never been, but I’ve heard of it. That’s great. |
| 1:45 | Amy: It’s warm. It’s a lot warmer than Minnesota, which is where I live most of the year. |
| 1:52 | Francesco: We have refugees from Minnesota down here too. It seems like their — a lot of folks escape to the South and Southwest. I don’t blame them. Thanks for joining us again Amy. |
| 2:04 | Amy: Thank you so much. |
| 2:06 | Francesco: You’re welcome. Next up is Karen. |
| 2:14 | Can you hear us Karen? |
| 2:22 | I’m going to shout out to her in the chat here. |
| 2:35 | Okay. Well, maybe Karen will chime in later on. |
| 2:39 | Let’s go to Kate. |
| 2:54 | Kate, if you can hear us, we’re doing introductions and I invite you to introduce yourself. |
| 3:05 | Please make sure to check that your microphone is green. |
| 3:10 | Kate: Can you hear me now? |
| 3:11 | Francesco: Now we can hear you. Yes. Thank you. If your microphone is orange. |
| 3:13 | Kate: Okay. I was clicking on… |
| 3:15 | Francesco: Ah. I see. |
| 3:15 | Kate: Yes. I was clicking on it and thought it was going to unmute, but I guess it finally did. So, hello everyone. My name is Kate Saintcross. I started working with Excelsior in 2008 in the Admissions area and moved to the Business and Tech area for advising. Currently, I have been an adjunct faculty since 2014. |
| 3:39 | Francesco: Great. What do you teach? |
| 3:41 | Kate: I’ve recently been teaching the upper-level management course for the business area in the School of Undergraduate Studies, and I’ve taught organizational behavior in the past as well, which I love. |
| 3:55 | Francesco: Great. Are you in Albany? Are you close to campus? |
| 3:59 | Kate: I’m in the Albany area, yes. |
| 4:00 | Francesco: Awesome. Okay. Well, welcome. |
| 4:03 | Kate: Thank you. |
| 4:05 | Francesco: I see that Karin’s microphone has lit up. Karin, can you hear us? Do you want to introduce yourself? |
| 4:15 | Okay, not yet. So, sometimes when you login to Go To Meeting, it automatically mutes you, and you’ll know because the microphone will be orange. So, if you click on the microphone icon, it should switch over to green, which means that we can now hear you. |
| 4:30 | Next up is Paul. Please introduce yourself. |
| 4:33 | Paul: Hi. |
| 4:34 | Francesco: Hi Paul. |
| 4:36 | Paul: Can you hear me? |
| 4:37 | Francesco: I can. |
| 4:39 | Paul: Oh great. First of all, I’m an adjunct instructor teaching history, mostly world history. I have been with the college as an adjunct for… Well, I can’t even remember. I started adjuncting there as a test grader on their tests for credits and then switched over to online teaching. I’m a SUNY retiree. |
| 5:13 | Francesco: Okay. Which SUNY? |
| 5:17 | Paul: SUNY Empire State College, what used to be the Center for Distance Learning. |
| 5:24 | Francesco: We have a colleague who also left Excelsior, who now adjuncts for SUNY. So, she did the opposite. She’s at SUNY teaching English. Patti Croup. |
| 5:37 | Paul: Yes. |
| 5:38 | Francesco: So, what do you teach at Excelsior? |
| 5:41 | Paul: I’m sorry? |
| 5:42 | Francesco: What do you do at Excelsior? |
| 5:44 | Paul: I’m an adjunct instructor. I teach history courses, mostly world history and occasionally US history. |
| 5:54 | Francesco: Okay. I assume you are in the Albany area as well. |
| 5:57 | Paul: Yes. I live just across the river from Troy. It’s about 10 miles from the college. |
| 6:05 | Francesco: Okay. Great. Well, welcome. |
| 6:08 | Paul: Thank you. |
| 6:09 | Francesco: Sandye, you’re up next. |
| 6:13 | Sandye: Hi Frank. How are you doing? |
| 6:14 | Francesco: I’m doing well. Thank you. |
| 6:16 | Sandye: Good. So, I’ve been teaching for Excelsior for about nine years. I teach primarily biology and microbiology and developmental biology. |
| 6:26 | Francesco: Okay. Are you in the Albany area too? |
| 6:32 | Sandye: Oh no. I’m in Montana. |
| 6:35 | Francesco: Montana. Wow. I love Montana. Which part? Which part of Montana? |
| 6:40 | Sandye: I’m just west of Missoula. I’m about an hour away from the Idaho border. |
| 6:46 | Francesco: Okay. I think that was… I did a road trip — my wife and I did a road trip about 10 years ago, and I think that was the way we entered the state from Idaho. |
| 6:58 | Sandye: Oh. Sure. |
| 6:58 | Francesco: So, we probably passed through there. |
| 7:01 | Sandye: Absolutely. |
| 7:02 | Francesco: I understand why they call it big sky country now. |
| 7:06 | Sandye: Absolutely. |
| 7:07 | It’s impressive. |
| 7:09 | Francesco: Welcome. And finally, last but not least, Susan. |
| 7:14 | Susan: Hi everybody. I’m Susan Honea. I live just north of Atlanta, Georgia. I am a professional writer and copy editor, and I’m adjunct at Excelsior. |
| 7:23 | I teach primarily in the English department in the professional and technical writing concentration and occasionally also in the School of Business. |
| 7:35 | Francesco: Thank you for joining us again Susan. Susan’s been a stalwart with the series, as have some others here. |
| 7:41 | Karin, if you can hear us now, if want to introduce yourself, go ahead. Otherwise, maybe we’ll catch you later on in the webinar. |
| 7:55 | I wanted you to introduce yourselves and what you do so that we can get a sense of the variety of disciplines represented here too. |
| 8:01 | We’ve got people even teaching in multiple disciplines: English and business, business courses and faculty management, history, biology, English. |
| 8:12 | We’ve got some wide representation here in different disciplines, different schools. |
| 8:21 | This is a Writing in the Disciplines Training Series. |
| 8:25 | I think it brings a good perspective to the topics we’ll be discussing. |
| 8:29 | Some of the things that we discuss in this training series are kind of universal when it comes to teaching writing, but we also want to emphasize the way they get articulated differently in each of the disciplines. |
| 8:41 | You’ll see that when we get to the rubrics, we’ll be talking about… Actually, I’ll be showing you some sample rubrics from some different disciplines. |
| 8:48 | So, let’s move forward. |
| 8:53 | In this overview, you can see we’ve split up the presentation into two parts. |
| 8:57 | The first part will focus on effective feedback and we’ll look at… We’ll talk about what is the purpose of feedback, challenges for providing feedback, some dos and don’ts, some feedback modes, and then some tips for how to give feedback and grade more efficiently. |
| 9:14 | That will lead us into a discussion of rubrics. What is a rubric? Why use rubrics? We’ll look at some different types of rubrics and then talk about some design tips for creating rubrics. |
| 9:27 | Finally, we’ll think about using rubrics to evaluate institutions and improve institutional effectiveness. |
| 9:36 | All right. So, Part I – Effective Feedback. |
| 9:38 | So, what is the purpose of feedback? |
| 9:42 | Feedback—we’ve all given it; we’ve all done it. I’m sure we’re all constantly trying to improve the way we do it. |
| 9:49 | If we had to define it—I think that this is a pretty good working definition here—it’s to monitor learning, assess performance, and offer suggestions for improvement. |
| 9:59 | Often we put the emphasis on the last two there or at least last one. You know, we want to give them suggestions for how to improve. |
| 10:07 | There are two main categories that we often talk about when we break down feedback: formative and summative. |
| 10:14 | I remember when this was new to me when I got into the world of assessment with different projects. People would talk about formative and summative and I’d sort of just shake my head and go what is that? What does that mean? |
| 10:24 | Formative feedback is on-going feedback. It’s in the moment. It’s meant to assist students while they’re actually working on the task. They’re in process, and you’re giving them feedback so that they can be able to reflect and do better and improve as they’re doing the work. |
| 10:41 | Summative feedback, on the other hand, is more of an evaluation of the student work after its been turned in. So, it’s end of the day. The project is in. You give summative feedback. Usually, there’s some kind of grade attached or something like that. |
| 10:55 | These are the two main categories that we talk about when we think about feedback. |
| 11:01 | Within there, though, there can be sub-categories. This is talking about strengths and weaknesses. So, we give feedback. |
| 11:11 | Discrepancy feedback is feedback that’s aimed at identifying what’s wrong or amiss. |
| 11:16 | Progress feedback is aimed at what is strong and going well. |
| 11:21 | We can give discrepancy and progress feedback as part of formative feedback while the project is underway and as part of summative feedback after the project has been turned in. |
| 11:36 | I want to pause here and invite you to a dialog here. What challenges have you faced when trying to provide feedback? |
| 11:53 | Sandye: Frank, I think my biggest challenge is I don’t know if students are actually paying attention to it. |
| 12:00 | Francesco: Ah. That’s a good point Sandye. What leads you to question whether they’re paying attention to it? |
| 12:10 | Sandye: They make the same mistakes on every discussion every week. Or the same mistakes on every paper. |
| 12:20 | If they don’t open… |
| 12:21 | I spend so much time filling out the rubric and giving them links to OWL or to APA formatting, and I feel like they’re not using it. |
| 12:35 | Francesco: Do they have an opportunity to use it in a revision or are they always moving on to the next assignment? |
| 12:43 | Sandye: Always moving on to the next assignment. |
| 12:47 | Frank: It may help to… It’s basically summative feedback that you’re giving them. |
| 12:52 | Then they move on to the next discussion board or paper. |
| 12:57 | That might be something to think about is how can you scaffold it so that they’re sort of required, or at least would benefit from responding to that feedback. |
| 13:10 | Anyone else want to respond to that? How do we know if they are using it? I think they’re not using it. |
| 13:17 | Any tips there? |
| 13:19 | Paul: Francesco? |
| 13:20 | Francesco: Yes. Is that Paul? |
| 13:21 | Paul: Yes. It’s Paul. Yes. I have run into the same problems that she had mentioned. Students just look at the grade. |
| 13:31 | Francesco: Yes. |
| 13:33 | Paul: I think the only time that they might look at the feedback is if it’s like a C or a D on that. |
| 13:40 | Francesco: Yes. |
| 13:43 | Paul: Like she said, you see the same problems assignment after assignment after assignment. |
| 13:51 | Francesco: Yes. |
| 13:53 | Paul: You begin to wonder after a while if they’re taking a quick look at it. |
| 13:58 | Another thing I’ve noticed too is that students will just take very cursory looks at the directions for a written assignment. |
| 14:11 | What you’ll get is the right introductory sentence, then fail to see how many primary sources were you expected to use, etc. |
| 14:28 | Francesco: So, a similar experience there and then also them not reading the assignment. |
| 14:32 | I just want to acknowledge Sandye, I’ve had the same issue where again and again I’m giving the same feedback to students. |
| 14:43 | The only thing I can think of is to create a situation where they have to respond to the feedback because, like Paul said, I think they often just look at the grade. |
| 14:56 | The feedback to them doesn’t matter. It’s over. They’re done. They’re moving on to the next one. They don’t really care or see the purpose of reading the feedback. |
| 15:05 | That’s discouraging especially when we spend so much time on it. So, I definitely feel you there. |
| 15:11 | Anyone else on here with challenges of providing feedback? |
| 15:16 | Kate: This is Kate. Some of the experiences that I’ve had with providing feedback have been very effective. |
| 15:26 | I think it just really depends on the students. I think some students want to get the work in, get the work out, get the grade, and move on. |
| 15:34 | But there are some students, I think, who are genuinely interested in their writing skills. |
| 15:40 | What I do is when I provide feedback—and like I said, I share the same experiences that some of you’ve had with some students don’t seem to pay attention—but when I am grading their work, I take notes along the way. |
| 15:55 | When I see students making progress, then the next time I’m giving them feedback, I let them know that their grammar has been better. |
| 16:06 | I try to let them know that their work seems more masterful, and they are communicating their great ideas in a more effective way. |
| 16:19 | Some students just pick up on that, and they just keep going. Some students have even said that they’re excited to know that their writing is catching up with the ideas, in a sense. |
| 16:30 | I don’t think it’s always effective because I think some students just look for the grade that they get and they don’t read all the feedback. So, I think that’s just part of how it is. |
| 16:41 | Francesco: Thank you for sharing that. |
| 16:42 | I think that can be powerful because it shows the student that you’re paying attention, that you remember, and it acknowledges the change that they’re making. I think that’s really powerful. |
| 16:58 | Who’s up next? Anyone else want to share challenges that you’ve faced when providing feedback? |
| 17:04 | Amy: Yes. This is Amy. I’m curious. I wonder if people do this all ready, but in the little notes section underneath the rubric, I typically write their name and say please see attached for feedback, but I’m wondering if it would be better—because I feel like I have the same situation where I’ve told students again, and again, and again to do certain things—instead to say something like, “Please see attached for important feedback that will help you on your upcoming assignments” or something like that. |
| 17:40 | Something a little more like you do need to look at this. |
| 17:44 | Francesco: Yes. |
| 17:45 | Amy: You know what I mean? If there was something like that. It’d be interesting, just even anecdotally, to see if we all did that for the next four weeks, would we see any change? |
| 17:56 | Francesco: That’s interesting. Anyone want to comment on that? |
| 18:01 | Susan: Yes. This is Susan. I don’t necessarily think it would make any difference at all. |
| 18:12 | I teach at three different levels within the college currently. I have your typical 101 students, and then I have the mid-range students, and then I have the ones who are about to graduate. |
| 18:24 | It doesn’t really matter how much feedback I give, how it’s phrased, whether it says you have to do this, etc., in the scaffolded assignments that are built in to each of these courses that I’m teaching, I very rarely see a comprehensive revision at the end. |
| 18:42 | I’ve asked students about that. One of the things that they say is that they don’t have time. |
| 18:50 | That’s also a problem that we face as instructors in giving comprehensive feedback because it does take time. |
| 18:57 | But I think we have to get to a point where we’re creating a conversation with our students and we are causing them to see the relevance of these revisions in the bigger picture scenario, and how it would affect them out in the real world. |
| 19:15 | One of the things I really like about Canvas—and to a certain extent the other learning management systems do this as well—you can create a writing conversation or a writing conference, so to speak, within the feedback area because the student has the ability to respond to you and ask questions. |
| 19:37 | So, occasionally what I’ll do is I’ll throw a question in there and see how many of them respond. |
| 19:43 | Then, I’ll send out an inbox message to say, “Have you checked your feedback this week? There’s a question that needs to be answered.” It kind of creates that Oh I guess I need to go look at that. |
| 19:54 | Secondary to all of that is the challenge that some students have with even seeing the feedback that we provide. |
| 20:01 | I have numerous students, especially those who are overseas who have limited internet connectivity and those sorts of things, tell me that they cannot see the detailed line item comments that are included in the document itself. |
| 20:16 | So, I think there are a number of different challenges here that we have to face, and I certainly don’t have the right answer, but this kind of conversation is definitely a good starting point. |
| 20:31 | Frank: Thanks for sharing that Susan. |
| 20:34 | You know as we’re talking about this, I keep thinking about an experience I had in an English 100 course where the students were required to do multiple drafts and that was one of the only times where I felt like they really benefited from my feedback because it was by design. |
| 20:55 | You know if they wanted to improve their grade, they had to respond to that feedback. |
| 21:00 | I think that the point about them being busy I think is really important. |
| 21:06 | I teach for Excelsior regularly, and I know how overloaded sometimes the courses are. |
| 21:11 | Every week there are two or three new assignments that are graded. I could see how that would even discourage them from reading the feedback since they’re just going to move on anyway, right? |
| 21:24 | I wonder if there’s a kind of design solution to feedback. To making it more effective. Whether scaffolding or something else. |
| 21:39 | Interesting conversation. Any other thoughts on this? |
| 21:43 | Otherwise I can shift into some dos and don’ts here for the feedback itself, but I think we’re talking at a meta level here. Once we give them this wonderful feedback, how do we know that they use it? How do we know it’s benefiting them? |
| 21:55 | That’s important. That’s a really important question. |
| 21:58 | Amy: Francesco, could I ask a quick question? This is Amy. |
| 22:01 | Francesco: Sure. |
| 22:02 | Amy: I just wondered if you, or any of the other faculty online, have ever used the audio on the feedback. |
| 22:11 | Francesco: I haven’t, but people… In the last webinar, people were really recommending that, and I’ve heard other people recommend that—that it can be more personal and maybe even be more efficient. Take less time. |
| 22:22 | Susan: It’s not initially, but it can be with practice. I’ve done it. Some students love it. The challenge is that from an ADA point of view if you include an audio you also have to include a transcript. |
| 22:35 | Amy: Okay. |
| 22:36 | Susan: That’s an ADA compliance rule, and there’s no getting around it. Anybody who receives federal funding has to meet that requirement. |
| 22:46 | Amy: Makes sense. |
| 22:49 | And that doesn’t automatically populate? |
| 22:53 | Susan: No. There are several tools out there that will do it for you. It doesn’t necessarily do it 100 percent correctly, but it’s easy to fix it. It’s definitely an option, and students have told me that they really like it. I’ve done it over the years. |
| 23:13 | Sometimes what I’ll do is I’ll ask the student, and I’ll have it documented, I’ll say, “Would you prefer audio feedback, written feedback, or a combination,” and if they respond that they want audio feedback, then I ask if they require a transcript. I’ve covered my derrière so to speak. But if you want to be by the book, the transcript is supposed to be included. |
| 23:36 | Paul: Susan? |
| 23:37 | Amy: So, it doesn’t do that automatically. |
| 23:40 | Paul: Susan. This is Paul here. Is that capability available through Canvas where you can do an audio? |
| 23:53 | Francesco: It is. |
| 23:55 | Susan: It is. |
| 23:55 | Francesco: I think Canvas has voice to it, but I don’t know if it does transcription. |
| 23:59 | Susan: It’s called Speech Recognition down at the bottom. Well, there’s a button that says Speech Recognition. In between… |
| 24:06 | You know where Paul the attachment button is down at the bottom? There are three buttons together. |
| 24:10 | Paul: Yes. |
| 24:12 | Susan: The first one is the attachment. The middle one is the media comment. That’s where you can record yourself. Then, there’s also speech recognition. |
| 24:20 | I haven’t played around with the speech recognition piece, but I wonder if it would do it automatically. I’ll have to play around with that. |
| 24:27 | Paul: Okay. Great. Thank you. |
| 24:31 | Francesco: I’m going to move us forward here as I see that we’re almost at the middle of the hour, but this has been a really good conversation. |
| 24:40 | All right. So, let’s look at some Dos and Don’ts here. |
| 24:42 | There’s a lot of information here and, by the way, we’re going to provide this as a handout once all of this information will be up on the OWL, we will have the handouts up there as well. |
| 24:54 | On the left there in green, we’ve got the Do’s, and they’re matched to the corresponding Don’t on the right. |
| 25:00 | I’ll just go down the list here. |
| 25:03 | There’s a tendency, I think, for faculty to try to mark up everything, right? Let’s give them all the feedback I can provide. It’ll take an hour and grade this thing and really mark it up, and we feel like we’re doing such a great job for the student. |
| 25:16 | But, actually, identifying everything to revise works against the student because it tends to—and you know there’s research on this—it tends to diminish their motivation. They feel overwhelmed by all of that red ink. |
| 25:30 | The Do here is to limit the focus. Look for patterns of errors, even focus on a couple patterns at a time. Maybe this paper focuses on these patterns and the next one go to the next set of patterns. |
| 25:42 | Approach it from a global perspective. Look at global concerns. |
| 25:48 | Prioritize the learning objectives, whatever those happen to be for this assignment. |
| 25:52 | On the other hand, do not overwhelm the student with too much red ink. |
| 25:57 | Your job is not to proofread or copy edit the paper. We’re helping them to learn, and that’s going to be a process. They’re not going to get it right with the next try. It’s going to take a few tries. |
| 26:09 | So, focus on a couple different things each time. |
| 26:12 | The second tip here is to encourage problem solving. Don’t try to solve their problem for them. |
| 26:18 | I think it’s really hard to do. |
| 26:21 | Here we’ve got some canned comments that you might write. |
| 26:28 | On the don’t side, you might write something like, “You should do this” or “You need to do that.” In other words, you’re kind of telling them what they need to do. |
| 26:34 | As opposed to writing comments that lead them to think about a solution. “How could you clarify this here?” or “What’s another way to say this?” |
| 26:42 | And then they might even think of a way to say it that’s better than they way you might have thought to say it. |
| 26:47 | So, encourage problem solving. Tip them off that something needs to change here, but ask them to think about what the solution is. |
| 26:56 | The next one here I think is really critical. You know sometimes in our language we tend to talk about student writing in a way that focuses on their innate ability. |
| 27:07 | To say things like, “You’re so good at this!” “You’re so good at using evidence.” Or, “you don’t understand how to do this.” |
| 27:19 | It can come off as if we’re sort of evaluating the student him or herself rather than the task. |
| 27:25 | So, the tip here is to focus on the task. Use language that focuses on the task itself. “This is a very smart introduction. I like XYZ about it.” |
| 27:35 | That that gives them feedback that they can then operationalize somehow. |
| 27:40 | Now they know these are the things they’re good at or not good at, and they can try to work on those things. |
| 27:46 | Refer to your Handbook for help with XYZ. |
| 27:49 | Again, it’s the constructive feedback. |
| 27:52 | To me this reminds me a lot of Carol Dweck and the mindset research where we don’t want to essentialize the student and treat them as if they’re an A student or a C student. |
| 28:06 | But to say, “You’ve worked really hard here and it shows in these areas.” That suggests that they can—through continuing to practice—they can improve. |
| 28:16 | Be specific. Of course, that’s always a good tip for providing feedback. |
| 28:21 | I know I’ve been guilty of giving vague feedback in the past. The thing not to do. Sometimes I’ll read something and be flabbergasted and put an exclamation mark. I remember doing that. |
| 28:32 | Now that I think back on it, I realize well, that probably wasn’t very helpful because they didn’t know what was in my mind when I wrote that. |
| 28:38 | So, saying things like “awk” or “good work,” that doesn’t really tell the student much. That’s vague. |
| 28:44 | Instead, be specific. Say things that really pin it down like, “Oh, excellent use of evidence here.” Or, “you really did a great job of introducing the source in this paragraph here.” |
| 28:55 | Or, if they need help, you might say, “what do you mean in this part?” Have it written right there annotated in the margin. |
| 29:03 | Finally provide timely ongoing feedback. |
| 29:07 | That means you can use different methods. It doesn’t have to be just one method. |
| 29:10 | The feedback doesn’t only have to come from you. |
| 29:13 | You can have built in peer and self-assessment. |
| 29:15 | We talked a little bit about that in the last webinar where we looked at peer review. |
| 29:20 | We also, I think, talked a little bit about having the students reflect on their work after they get it created. |
| 29:27 | You can do teacher conferences. |
| 29:28 | You can point them to digital resources like the OWL. I know a lot of you use the OWL and other sites. |
| 29:33 | So, there are different ways for them to get feedback and support. |
| 29:42 | I just want to work through couple of different feedback modes here, and I think we’re aware of this and some of this came up. |
| 29:48 | One is verbal feedback which can be provided synchronously, you know face to face. |
| 29:54 | For online courses, maybe you have a video conference. |
| 29:57 | Or you can also provide it asynchronously like we were just talking about. |
| 30:00 | Some kind of voice threading or voice memo, on a paper and they can get verbal feedback that way. |
| 30:06 | Written feedback. I know obviously especially working online that is the number one modality. Providing written feedback through an LMS or whatever technology you’re using. |
| 30:18 | Workshopping feedback which includes peer reviews peer feedback. That’s another mode of feedback. |
| 30:25 | The last one here is using rubrics. We’re going to get to that in a moment. |
| 30:30 | First, here’s some tips for grading more efficiently. |
| 30:34 | We all wish we can spend infinite time on a paper but actually it probably doesn’t help the students as we said. Don’t overwhelm the students and don’t do it for them, right? |
| 30:45 | If you see a pattern of error, correct one and then let them find the rest. |
| 30:50 | A good tip here is to use a timer. How many of you use a timer when you’re grading? Does anyone use a timer? |
| 31:05 | No? So, no one uses a timer? |
| 31:07 | I got into the habit of at least timing myself, if not using a timer. I found that very helpful. So, just a certain number of minutes that you’re going to spend on each paper. |
| 31:19 | That will help you. You’ll see that you’ll become way more efficient that way. |
| 31:27 | Type or speak your comments. Again, that came up earlier. |
| 31:32 | Speaking the comments is definitely something that I am interested in trying at this point. I’ve heard some good things about it. |
| 31:38 | As Susan said, maybe it’s a lot of work up front, but then you get into it and have a system and then it’s more efficient. |
| 31:45 | Create a Bank of saved comments to reuse. |
| 31:50 | Also have links to resources there. |
| 31:53 | Actually, I just had a conversation with some folks at another college, University of Maryland, where they do this. |
| 32:02 | They have a bank of saved comments that they initiate through an auto correct feature in Word. They’ll start to type a word, and it’ll bring up these saved comments. |
| 32:11 | The comments will have a link to some resource, usually the OWL. |
| 32:16 | Let’s say the comment… |
| 32:18 | These would be for things that you are commonly correcting in student papers like, let’s say, comma splices or sentence fragments or the introduction of evidence or citation formats. |
| 32:34 | Things that come up often. The thesis statement. Comments about an introduction or an abstract. |
| 32:42 | You might have just some pre-generated comment that maybe is initiated through some kind of auto correct feature and includes a link to a resource. |
| 32:51 | Finally, if you want to grade more efficiently, rubrics help to grade more efficiently. |
| 32:57 | I want to get into the rubrics right now. We’ll talk about what they are, how they work, and why they are efficient. |
| 33:05 | So, what is a rubric? The first thing to comment on here is that rubric actually has many definitions and it means different things in different fields. |
| 33:15 | In academic context, a rubric is a mode of feedback that’s typically very structured and as a result is more efficient for grading. |
| 33:24 | Rubrics are often used for grading or evaluating. |
| 33:28 | They’re very structured. For instance, they will have a list of criteria. They will break down the score by criteria for assessing that assignment. |
| 33:41 | Why use rubrics? |
| 33:43 | First, let me just ask does anyone here not use the rubric? |
| 33:52 | I figure if you’re teaching for Excelsior, you’re familiar with the rubric since those are pressed into our courses. |
| 33:59 | So, why use rubrics? |
| 34:00 | For students, rubrics are really good because they provide a clear set of expectations for the assignment. They can just look at the rubric, the grading rubric, and they know what they’re going to be evaluated on. |
| 34:10 | They can see the criteria. They can see the different scores and what they need to do for each criterion to achieve the maximum score. |
| 34:22 | So, it just makes the whole process more transparent for them. |
| 34:26 | They can use it like a checklist to make sure that they’re completing parts of the assignment, especially if the rubric is written clearly and written with the assignment in mind so that the learning outcomes for the assignment are part of the rubric. |
| 34:40 | For faculty rubrics, as we said earlier, they can make grading faster and easier by offering that simple format for conveying feedback. |
| 34:48 | So, instead of doing a grade at the end and then like a two paragraph explanation, or a two page explanation, you’ve got the rubric and it’s all mapped out. Then you just check the boxes and then maybe provide a comment in addition. You can also provide comments on each of the criteria. |
| 35:06 | Rubrics also make grading more objective and consistent, or at least appear that way. |
| 35:12 | Sometimes that helps to smooth things with students so that they feel like it’s fair, everything is fair. |
| 35:18 | For programs and institutions, rubrics are important as well because it makes it possible to compare student work across different sections, different courses, and even different institutions. |
| 35:27 | That way you can collect data. You can look at large sets. |
| 35:31 | So, rubrics are really important at the institutional and program level. |
| 35:37 | There are different types of rubrics, and I’m going to talk about two of them here. |
| 35:42 | One is holistic rubric, and it’s the one we all know and may or may not love. |
| 35:48 | Growing up, the most common way of my work was assessed was through a holistic rubric. I’d usually get an A, a B, a C, a D, or some plus minus version. Hopefully not an F. |
| 36:00 | That’s a holistic rubric. |
| 36:02 | Each one of those grades has a description that sums up the characteristics of the paper. |
| 36:09 | There’s an A-level paper, a B-level paper, a C-level paper, etc. |
| 36:15 | So that’s a rubric; and that’s a holistic rubric. |
| 36:19 | Now, since becoming an educator, and especially since teaching at Excelsior, I’ve learned to use the analytic rubric instead. |
| 36:30 | Analytic rubrics are—that’s what I was describing earlier—where you have a set of criteria, so instead of having a single score that sums up the paper, you have the requirements for the paper broken down by the different criteria and then each criteria gets scored with some kind of rating system. |
| 36:52 | Often that rating system has a description attached to each of the scores so that you can see why you got that score. |
| 36:59 | This format is more itemized, and it breaks down the assignment into the component parts. |
| 37:05 | You can see why this would be a little bit more explanatory for the student because now they get a better understanding of why they got the score they got. |
| 37:14 | Maybe there’s a column at the end where you add up the score for each of the criteria. You get to add it together, and then they have their final score. |
| 37:21 | I want to pause here and just ask what is your experience using rubrics? Also, you can think about what’s your experience with having rubrics used with you. Like, for instance, being a student and seeing a rubric for the first time. |
| 37:36 | So, what is your experience with rubrics? |
| 37:43 | Paul: Francesco? Paul here. As an instructor, I like them for the reasons outlined here. |
| 37:52 | Sometimes in the individual areas in the analytical one, for example, at the A-level, it’s almost like the student is handing in something that’s publishable on that, which is rarely the case, or has come up with some brilliant new theory or so, but there are ways of fitting in what they’ve done into it. |
| 38:25 | As the student, unfortunately in my time as an undergraduate, even graduate school, it was just what the instructor wanted to put down on your paper when he or she handed it back to you. |
| 38:47 | Francesco: So, as an instructor, you find that they’re really helpful. |
| 38:50 | Paul: Oh yes. And it makes things go much faster in evaluating papers. |
| 39:01 | Francesco: Okay. So you find it’s more efficient. |
| 39:05 | Paul: Yes. |
| 39:06 | Francesco: Okay. Anyone else? What’s your experience good, bad, or ugly using rubrics? |
| 39:14 | Kate: Hi. This is Kate. I like the rubrics, and I’m glad they’re built into the course. |
| 39:23 | If I had a wish list, I would wish that the rubrics could align a little bit more closely with the expectations of the assignments. |
| 39:33 | For example, in one of the modules there’s an assignment that’s required. |
| 39:43 | The rubric… One of the criteria on the rubrics is to grade an introduction, but that’s not expressed in the assignment area. |
| 39:56 | So, I need to communicate to the students what’s expected, to augment that. |
| 40:03 | But if rubric was more closely lined up to— |
| 40:06 | If each of the rubrics were more closely lined up to the expectations of the assignments, I think that would be even more helpful. |
| 40:13 | I do like the rubrics overall. |
| 40:15 | I think that helps to keep the grading fairly objective. |
| 40:20 | So, that’s my take on it. |
| 40:22 | Francesco: Okay. |
| 40:25 | Thank you. I can see how it could be more objective and also aligning it to the specific requirements or outcomes for the assignment might be a little better too. |
| 40:37 | Anyone else? Your experience using rubrics. |
| 40:51 | Amy: I have kind of an interesting experience I just had with another school. |
| 40:56 | Their rubrics you select the box that corresponds to their capabilities in an analytical rubric and what it does is it populates a video with the transcript of the entire rubric, of the lead instructor giving that feedback. Does that make sense? |
| 41:23 | Francesco: I didn’t quite understand. So, what’s the— |
| 41:30 | Amy: So, what the class has for the assignment, there’s a rubric, and you’d go ahead and tic the different boxes. |
| 41:38 | Let’s say if you go all the down on exemplary for all of them, what that does then is that populates a video of the instructor talking to all those different points, and then gives a written transcript of it as well underneath the video. |
| 41:56 | It’s pretty cool. |
| 41:57 | Francesco: Wow. |
| 41:58 | Amy: Yes. Maybe other people have had the same experience. |
| 42:04 | I’ll jump off, but yes, I love the Excelsior rubrics too for the exact same reasons we’re talking about here: making it consistent. |
| 42:11 | Some need to be a little more aligned, but for the most part, I think they’re really solid. |
| 42:25 | Francesco: Thank you. Was someone else about to speak? |
| 42:42 | Okay. I’m going to move forward then. We’ll have some time for additional comments later on. |
| 42:54 | Here we have some tips for designing a rubric. |
| 42:57 | For rubrics that are better intended for students, we recommend beginning with the discipline specific course learning outcomes. |
| 43:07 | What are they? |
| 43:09 | This is all about aligning the rubric to the assignment. |
| 43:15 | The assignment should be aligned to whatever the discipline specific outcomes are. The rubrics should be aligned to that. |
| 43:23 | Use language and terminology that students will understand. I think sometimes that we can be guilty as faculty of being overly technical, jargonistic, and we need to remember that our students are learning to be part of the discipline. |
| 43:38 | They’re not there yet, so they may not understand some of the ways that we communicate. |
| 43:45 | Be aware of that and use language that they’ll understand. |
| 43:49 | Include language about discipline-specific conventions, such as citation style. |
| 43:53 | So, if you want them to use APA, let them know that’s the style of citation they’ll be using and that also APA follows a certain format or organization. Maybe even give them a link to some resources on that if they’re not aware of that. |
| 44:11 | Break down the assignment expectations into clear, discrete elements. This is that itemizing, right? |
| 44:16 | So, what are the clear, discrete elements? |
| 44:19 | How many should you have? |
| 44:20 | I think probably 10 is too much, but maybe five? |
| 44:27 | Then think about the weight each of those elements has because they don’t all have to be weighted evenly. |
| 44:32 | I think the weight shows the student which areas are most important for assignment. |
| 44:40 | So maybe for one assignment, critical thinking is most important, but for another assignment maybe it comes down to use of language, word choice, paragraphing, structuring, and so on. |
| 44:58 | These are different types of assignments or even steps along the way towards a final product. |
| 45:06 | Offer criteria or examples for each of the different levels of achievement. |
| 45:12 | So, what does it mean to exceed expectations, or get an A, or score a 6? What does that mean? |
| 45:20 | Don’t just have a score; have a description for that score for each of the criteria. |
| 45:28 | This is an interesting thought here: involve the students with creating the rubric. I’ve never done that before, and I’m really interested in doing this. |
| 45:38 | I think if there’s time and flexibility, I think that could really work well, especially from the perspective of creating more buy in. That’s often a hurdle we have with students. |
| 45:49 | Motivation. They don’t see the point or the purpose of whatever assignment they’re doing. I think that would help them to sort of buy into it more and also understand it better. |
| 46:01 | Now if you’re designing an assessment rubric, I think a lot of the same things apply. |
| 46:06 | One thing that we wanted to share was to use precise, technical terminology. |
| 46:11 | You want to ensure that there’s consistency and there’s reliability between multiple graders. |
| 46:17 | So, you want to be really clear and concise with your language when designing these rubrics. |
| 46:25 | We’re going to look at an example of an institutional rubric later on. |
| 46:31 | Let’s get into some sample rubrics here. This one here is for a psychology assignment. It’s an annotated bibliography assignment. |
| 46:40 | You’ll see that on the left there we have the different domains or criteria. |
| 46:45 | They’re weighted differently. |
| 46:49 | Then we have the different scores that they can get from Excellent to Fails. |
| 46:56 | Beneath each of the scores there’s just a short description of what the student has to do to achieve that score. |
| 47:03 | In that sense, it’s really helpful. |
| 47:05 | It’s helpful to know that if you want to get excellent in the number of citations category, which is a 30 percent chunk of your grade, that basically, you have to cite 10 sources. |
| 47:17 | Also, those sources, the quality matters, so they need to come from psychology journal articles. |
| 47:25 | Again, it’s just a little bit more descriptive and provides that student with that kind of a checklist where they can go through and see exactly what they need to do. |
| 47:35 | Here we’ve got a little bit more verbose rubric. |
| 47:38 | This is for a nursing case study. This is a graduate assignment, and it’s only part of the rubric. |
| 47:44 | It’s actually much longer and has more criteria, but you kind of get the picture here. |
| 47:49 | Again on the left, you’ve got the criteria, a description of each of the criteria, and the weight for each of the criteria. |
| 47:57 | Then on the right, you’ve got the different scores, each one with a description of what it means to get that score. |
| 48:04 | There are some key words underlined, which is, I think, helpful. |
| 48:09 | I’ve learned it’s helpful to use the same keywords down each column. |
| 48:14 | So, at the mastery level, you’ll see the same words recurring in each of the criteria. Something like that. |
| 48:24 | Again, it conveys to the student what the student needs to do for this assignment. |
| 48:29 | You’ll notice that the criteria are very disciple specific and very assignment specific. |
| 48:36 | They include things like patient characteristics or patient diagnosis or principles of pharmacokinetics. |
| 48:43 | You won’t see that in just a standard English academic essay rubric. |
| 48:49 | This is a way in which the rubric is designed to be discipline specific, even genre specific. |
| 48:58 | Here’s an example of a rubric for an anthropology assignment that is not a written assignment. This is an oral presentation. |
| 49:06 | On the left there you’ll see that their number of criteria 1-2-3-4-5-6-7-8 criteria each one with a short description, and we’ve got the scores again each one with a short description. |
| 49:20 | Then all the way to the right there’s a score column, and that can be tallied up. |
| 49:25 | So, this is clearly being used to grade students. |
| 49:31 | So, I want to pause here and just talk about digital rubrics for teaching online. |
| 49:36 | Now we have mentioned a couple of digital rubrics that we use in LMSs. |
| 49:43 | I know we’re all familiar with the Canvas rubrics that Excelsior uses. |
| 49:47 | Canvas provides a rubric tool to evaluate student work, and we’re all encouraged to use it. |
| 49:57 | These rubrics work by creating outcomes and performance indicators, and you can assign points and percentages. |
| 50:04 | Sometimes you can even have optional secondary rubrics. |
| 50:06 | So, if you’re teaching online and you have access to a digital rubric creator in Canvas or Blackboard or whatever, that’s a way to use a rubric. |
| 50:19 | Often those rubrics are pre-created which is good and bad. |
| 50:23 | It’s good that you have it and they’re aligned to the school learning outcomes or program learning outcomes. |
| 50:29 | Sometimes they may not be as aligned to the actual specific assignment and that may be something that you’ll want to think about. |
| 50:38 | There are also some free rubric generators out there that are customizable: Quick Rubric RubiStar, Essay Tagger, Rubric Maker. |
| 50:47 | These are different tools you can use to create digital rubrics. |
| 50:51 | Some of these rubrics are aligned with standard outcomes like the Essay Tagger. |
| 50:56 | Then there are pre-created rubrics out there that are mainly intended for institutional evaluation. |
| 51:06 | If you’ve heard of AAC&U, they offer a whole line of rubrics in different areas. |
| 51:13 | Here I’m showing you their written communication value rubric. |
| 51:17 | This is actually a rubric that we’ve used for the OWL to evaluate different pilot studies we’ve done because we were running multi-campus pilot studies with student samples from sections of different courses. |
| 51:34 | All of the courses were at the same level but at different campuses. |
| 51:37 | So, you need a rubric that is generalizable enough and universal enough to apply to these different situations. |
| 51:46 | The value rubric really fit the bill here. |
| 51:49 | They have additional rubrics. I know they just came out with a reading rubric to evaluate reading courses. They’ve got rubrics in different subjects. |
| 52:00 | Again, it just makes it easier to compare what otherwise my seem is like apples and oranges, but they’re not because in theory the students should be learning the same skills as they’re progressing through a course sequence whether it’s at Excelsior, or SUNY, or whatever. |
| 52:22 | I want to pause here. We’ve actually come to the end here and I just want to ask if there are any additional questions or comments that you want to share. |
| 52:43 | It’s a Friday afternoon, and I’m sure we’re all rearing to get to the weekend. |
| 52:53 | I want to thank you for joining us today. Enjoyed the conversation. |
| 52:58 | Just to let you know, we will be taking all of these training webinars and converting them into interactive content on the OWL as well as putting up a recording of each of the webinars for people who couldn’t attend. |
| 53:19 | We’ll also have some handouts available for each of the webinars for some of the key information in the webinars. |
| 53:27 | So that we’re hoping to roll out maybe by the end of this month, if all goes well. |
| 53:34 | You’ll be able to find it under Educator Resources. |
| 53:38 | So, stay tuned for that. |
| 53:42 | We do have links to additional resources in each of the webinars as well as the references that we’ve cited along the way. You’ll be able to expand your knowledge by following those links. |
| 53:56 | Otherwise, I just want to thank you for joining us for this fourth and final part of the Writing in the Disciplines Training Series. Again, it was brought to you by a collaboration between SUNY and Excelsior funded by a SUNY IITG grant. |
| 54:11 | I’m really happy that you joined us today. |
| 54:13 | Thank you. |
| 54:15 | [Other thanks and goodbyes.] |
