Category: Individual pages

Hoot! was launched in March 2016 to supplement the Excelsior OWL by providing a hub for educators to share ideas, learn from each other, and dialog. With Hoot, educators can: Explore topics related to the teaching...

Individual pages

October 4, 2021

Contact the Editor

Interested in being a guest contributor to Hoot? Hoot is sustained by the generosity and support of guest contributors who are passionate about the topics discussed in this blog. If you would like to...

Individual pages

May 16, 2019

Index of Activities

The Excelsior OWL offers more than three hundred multimedia activities designed to enrich the learning experience. There are more than a dozen different types of activities, including interactive video, multiple choice, and drag and...

Individual pages

May 16, 2019

Reading VALUE Rubric

Reading is “the process of simultaneously extracting and constructing meaning through interaction and involvement with written language” (Snow et al., 2002). (From www.rand.org/pubs/research_briefs/RB8024/index1.html) Evaluators are encouraged to assign a zero to any work sample...

Individual pages

May 16, 2019

Assessment Rubrics

Standardized assessment rubrics are used to evaluate individual student work, courses, programs, and institutions. The AAC&U VALUE Rubrics are some of the most commonly used rubrics. These analytic rubrics provide distinct domains to help...

Individual pages

April 11, 2019

Recorded Webinar

Individual pages

April 3, 2019

Recorded Webinar

Grumble... Applaud... Please give us your feedback!

About Transcript Handouts Additional Resources

A video of the live webinar held on February 20, 2019.

0:03	Francesco: Welcome everyone to the third installment of the Writing in the Disciplines Training Series.
0:09	In this section, we’re going to focus on peer review and revision, which is really a breakout from the last presentation where we talked about effective assignment design.
0:22	We ended by talking about scaffolding in that presentation. So, this picks up from where that one left off.
0:29	Just a bit of background—this is a project that is being co-developed with three SUNY community colleges: SUNY Rockland, SUNY Clinton, and SUNY Monroe.
0:42	All the content here is a collaboration between the writing across the curriculum and OWL people at Excelsior working with the writing across the curriculum and writing program folks at the three SUNY colleges I just mentioned.
0:58	We will be offering this both in webinar and on-site version at Excelsior.
1:04	There will be a fourth and final part which I’ll get to at the end of this presentation.
1:11	This is being supported by a SUNY Innovative and Instructional Technology grant.
1:20	I am Frank Crocco, the director of the online writing lab at Excelsior. I want to give you a chance to introduce yourselves. I will call out names and go through the list that way.
1:31	So, Amy, would you start us please? Introduce yourself?
1:36	Amy: Sure. My name is Amy Erickson, and I teach in the English department and I also teach in the business department, writing composition and humanities classes.
1:47	Francesco: Okay. Welcome Amy.
1:49	Amy: Thank you.
1:50	Francesco: And Bei?
1:53	Bei: Hi. My name is Bei Liu. I am a program director for Excelsior. I’m working on general technology programs, and it’s a very good opportunity for me to learn the writing across disciplines. I learned a lot from the first two sessions, so I’m looking forward to it. Thank you.
2:13	Francesco: Happy to have you. Thank you for joining us. Caroline?
2:19	Caroline: Hi. This is Caroline Mosca. I’m a faculty program director in the school of nursing. I am looking forward to this. We’re trying to integrate more writing in the disciplines into our courses. Looking forward to some more opportunities to do that. Thank you.
2:34	Francesco: Thanks for joining us. Grace?
2:42	Can you hear us Grace?
2:50	And if you for whatever reason can’t —don’t have a microphone or some way to hear us, please type it into the chat. Let us know.
3:03	I think we may be having audio issues there. Nicole, would you please introduce yourself?
3:11	Nicole: Sure. Hi. It’s Nicole Helstowski. I’m a faculty program director in the school of nursing. I primarily teach in the associate degree.
3:21	Francesco: Welcome. Richard, please introduce yourself.
3:26	Richard: I’m Richard Cole. I teach in the nuclear and general engineering technology programs.
3:33	Francesco: Great. Thanks for joining us. Susan?
3:37	Susan: I’m Susan Honea, and I teach in the English department and also in the Business department writing courses, particularly the professional and technical concentrations sequence and capstone. I’m also the subject matter expert for that concentration.
3:55	Francesco: Fantastic. Thanks for joining us Susan.
3:57	I’m happy to see that we have a nice mix of program directors and teaching faculty on the call today.
4:02	I want to encourage you all to participate in the conversation. I think that’s what really makes the workshop the most effective.
4:09	There are going to be points along the way where we’ll stop for discussion, or I’ll invite you for discussion. So, I’d just encourage you to jump in.
4:19	It’s a learning experience.
4:20	I believe in a concept called failing forward, so if you don’t know something, just ask. That’s how we grow together. I think we all have room for growth in these areas here.
4:35	Let’s get started here.
4:37	I’ll just share one more thing. There are handouts that will be produced from this content—that have been produced actually—and I shared those with you.
4:48	I can make those available again if for some reason you can’t find them.
4:53	We’ll also make this content all available on the Online Writing Lab very shortly.
4:58	Going to hop in here.
5:02	Okay. Just an overview of the presentation. As I mentioned, we’re building from the last presentation where we talked about scaffolding.
5:09	We’re going to focus on two specific parts of scaffolding here: peer review and revision.
5:14	We’ll talk about what is peer review, why do we do peer review, how do we do peer review, go over some examples of peer review assignments, and then we’ll switch to talking about revision: what is revision, why revise, how to revise, examples of revision assignments.
5:32	In there, we’ll also talk about some of the technology too that can help with that—looking at how to do it online.
5:39	Finally, we will end by looking ahead at what the next presentation holds.
5:45	This is the nice overview slide here.
5:49	In the last presentation, in fact we had the same slide, it showed you kind of an articulated process for scaffolding an assignment.
5:58	In this case sort of a standard academic essay.
6:02	So, you’d start with some kind of informal writing, or what we might call writing to learn type assignment: maybe a reflective piece, a discussion starter question, a discussion board post. Doing some informal writing.
6:16	Moving from there, to starting to identify a topic and doing some research and gathering information, maybe doing an annotated bib.
6:25	Then, once you have done a little research, do an outline. Start to structure what you’re going to write.
6:31	Then do the rough draft; put it all together in a rough draft.
6:36	Really it’s at that point that it’s pretty common to then bring in the peer reviewers.
6:42	Some faculty will grade the rough draft and provide feedback and some won’t.
6:47	Some will say bring it to your peers. Do some kind of structured guided peer review.
6:51	Get feedback from the students first and then maybe at that point come in and weigh in and give some feedback.
6:59	Of course, this feedback is all moving towards a revision stage.
7:05	It’s nice to give the feedback, but then it makes sense then to move forward to try to implement that feedback.
7:11	Do another revision of the draft and then turn it in.
7:15	We added there a final stage where there’s a stage for reflection where the author can reflect on this process and learn from it. Sort of identify what they can learn from it—articulate it for themselves.
7:26	So, what is peer review?
7:28	We’re going to focus on these two stages in the scaffolding process: peer review and revision.
7:33	What is peer review?
7:35	We had a pretty big conversation about it in the group that created this content.
7:40	We finally were able to whittle it down to the concept of quality control.
7:47	The idea is that when a student or a writer creates a first draft, there’s always room for growth. There’s always room for improvement—to increase, to improve the quality of that writing.
7:59	That’s when we turn to others to provide us with perspectives that we don’t have.
8:05	We get… The audience essentially comes in, receives this written work—because writing is always for an audience—and then we get feedback from the audience.
8:16	That is the point of peer review: to get that feedback so we have some kind of quality control mechanism in place.
8:22	During that process, we achieve certain things.
8:25	We review each other’s work in a structured and guided way—and I’m going to emphasize that—that’s part of making sure peer review goes well.
8:34	We provide feedback for revision.
8:38	As the reader of that work, we provide feedback to the writer.
8:43	But then, as the reader, we also learn how to evaluate writing, which is supremely beneficial because it helps us be stronger writers ourselves.
8:53	To be able to evaluate writing, to be a strong reader is part of learning how to be a strong writer. And that’s how often we learn to write well by reading things that are well written and also being able to analyze them as we read.
9:10	So, why do peer review?
9:15	From the student’s side, it helps the student get feedback from readers who are familiar with the expectations for that assignment.
9:24	These are other writers in your class who’ve all gone through the same assignment, and they can provide you with some targeted feedback because they are familiar with that assignment.
9:35	They have the same expectations and have had to deal with the same common obstacles and concerns as they were writing.
9:44	So, you’re getting that feedback from your peers.
9:48	It helps students to compare and contrast their own writing with other models.
9:52	They see other papers that other students have written and learn and identify ways that they could have constructed their paper and maybe helped them to come up with new ways to revise.
10:04	It helps them solidify their understanding that they’re writing for an audience.
10:09	In an academic situation, audiences are often missing. We sort of assume that the teacher is the audience, but that’s really artificial.
10:16	Here, your peers become the audience. It becomes much more grounded in a real-world conversation and sharing.
10:25	They are also practicing engaging in a disciplinary discourse.
10:30	Whether you’re writing in the sciences or math or liberal arts, you’re writing for a group of peers.
10:37	Whether it’s a journal article or a book or whatever, a blog, you’re writing for a group of peers in your discipline.
10:45	The students now become those peers and they take on the identity of being part of that discipline—whether it’s scientists, or mathematicians, or historians.
10:57	They’re engaging in that discourse as people becoming experts.
11:03	It strengthens their confidence in their own authority.
11:06	I think a lot of students have a lot of anxiety about their writing ability.
11:12	Putting them in that role where they’re a reader, I think, helps them to increase their confidence as a writer.
11:22	Skills that transfer. It helps them build skills for their future professional collaborations, teamwork experiences.
11:30	This is very much a collaborative experience where they are engaging with each other as peers. Reading each other’s work. Evaluating each other’s work.
11:40	This is the kind of work that we need to be able to do: giving feedback to each other in the workplace.
11:47	It’s also the kind of work that they need to be able to do, or the kind of skills they need for just daily life: conversation, collaboration, negotiation.
11:55	It’s really much more tuned to how we actually live our lives, than say, getting an assignment, going home, writing it by yourself, submitting it. It’s a very isolated process. And then revising it based on maybe feedback from the teacher.
12:09	Here, we are getting more feedback from the community, from your peers.
12:13	There are a lot of benefits for why doing peer review, especially from the student’s point of view.
12:19	Now, I want to be cognizant of the fact that students have reluctance to doing peer review.
12:28	In a minute we’ll share our experiences here, but some of the most common ones that they have—they have anxiety sharing what they consider incomplete work. “This is a rough draft. I don’t want others to see it.”
12:38	So that’s one anxiety they have.
12:41	I think it’s important to be aware of these anxieties as we assign peer review and develop peer review assignments.
12:48	They have a lack of confidence in their own authority. This goes back to them not having a sense of writerly authority.
12:54	They have a lack of confidence in the authority of their peers. “How do I know that my peers are going to give me good feedback?” So, they have uncertainty.
13:03	They also have uncertainty about the value of drafting and revision overall. “Why do I have to draft? Why do I have to revise?” “This is perfect as is.” “It came from my head onto the page, and it’s perfect. And there it is.”
13:18	We impress upon them that writing is an iterative process. It’s never over until we arbitrarily say it is.
13:24	Finally, they have confusion about how to apply peer feedback to the revision process.
13:29	So, they’re getting this feedback from students—even from us as teachers—but they’re not sure what to do with it. How do they implement it? That’s also part of their reluctance.
13:44	So, I want to pause here and invite you to share.
13:47	The first question here is what are your past experiences with peer review, good or bad, either as an instructor or as a student? As someone who may be in college, was asked to do peer review. Or in high school. Or wherever was asked to do peer review.
14:03	Also, as an instructor, as someone who has designed courses where there is a peer review component. What was your experience like when you went through peer review on either side there as the teacher or the student?
14:24	Nicole: Hi Frank. This is Nicole. Can you hear me okay?
14:26	Francesco: Yes. Perfectly. Thank you Nicole.
14:28	Nicole: Okay. I haven’t had experience as an instructor with peer review, but I have as a student.
14:37	Some of my experiences have been that I’ve found that my peers were kind of soft in their delivery for feedback. It was a lot of fluff, you know, “great job,” “really liked what you did.”
14:51	I know myself to know that I’m not that excellent of a writer. As a student, I was really hoping that the instructor would kind of step in and give some nudging to my peers to really give more formative feedback, so that I could really improve.
15:10	I wasn’t looking for a pat on the head. I was looking for more help.
15:15	And then as a student, knowing how to be able to navigate and ask for that. Like, “Thanks, but could you give me anything else?”
15:27	That’s been my experience in the peer review process. That’s tough when you know it’s not perfect.
15:38	I think sometimes peers aren’t apt to… They don’t know how to give feedback. They’re afraid of feelings.
15:48	It’s an additional challenge when you’re in an online environment. People can’t see your facial expression and your body language. It poses another challenge.
15:59	Francesco: Yeah. That’s a good point. So, when you went through it as a student, did you have… was it guided? Did you use a rubric?
16:12	Nicole: The rubric from the assignment. Yes. The feedback was not “take a look at the rubric.” It wasn’t directed like that, and now that you say that, I think that would have been really good for… It should have been built into that process.
16:27	Francesco: Yes. In a moment we’re going to look at rubrics too, at some different models of how to do that.
16:36	Thank you for sharing. Anyone else? Also, introduce yourself when you speak so this way we know who’s talking.
16:44	We will be creating a transcript for this webinar, so it just helps us to know who’s saying what.
16:52	Bei: Hello, Frank. This is Bei. I’d like to share. In one of my classes, normally we ask students to discuss and comment on other students’ responses.
17:04	So, one of [unintelligible] want to instead of writing their main post, mainly they have to respond to others and then give a wrap up discussion. I find that that’s very helpful to my students.
17:20	They’re doing even better when they do their own discussion post. That way, I kind of force them to read more of other students’ main posts and then collect the idea or information and then put in one.
17:38	It’s a [unintelligible] some of the student, especially in technology areas, if you give them one case study or one discussion topic, it’s easy for them to go to the internet, search it up, and put in their own discussion, but, when you check with Turn It In, it’s 50 percent, sometimes 80 percent, of similarities there because they just copy and paste. They didn’t process it.
18:03	When I ask my student to heavily comment on other student’s main posts instead of writing their own, then they have to process the information. I feel that’s helpful to the class.
18:19	Francesco: That sounds like it’s really helping the class. That sounds like a wonderful assignment. Thanks for sharing that.
18:30	Amy: Hi. This is Amy Erickson. I actually just finished grading a peer review this week.
18:35	One of the things I try to do prior to the peer review is to have a pretty honest conversation with the students, even though this is an online class, and say that your peers will really benefit from direct and honest feedback.
18:48	They don’t want to hear the fluff that I think Nicole discussed in her … that she experienced in the classroom.
18:56	I think that students will try to ease out of that assignment and just say, “Great job,” “The flow is fantastic.”
19:05	Instead I ask them to consider when they are also reading their feedback, that everyone in the class, the intent is to help each other.
19:15	And so, if you get feedback that seems strong or direct or never insulting, if they feel bad about the feedback to just remember the goal is for all of us to help each other to become better writers.
19:30	I try to lay that down before the peer review starts so students don’t have that kind of experience where they’re left feeling like the peer review was just not as rigorous as it should have been.
19:45	What made me do this was that I had some students just simply say—and I think the paperwork that we read prior to this meeting touched on this—“I just don’t want to judge anybody’s work.”
20:00	It wasn’t that they didn’t feel qualified, they felt it was something against their character to be judgmental about something.
2:08	I said well, then consider this as an evaluation. You know? If you don’t want to judge, then let’s evaluate this for how the student has put in to this paper the lessons that we’ve learned over the past six weeks. So, can you do that?
20:23	So, just some things that I feel have helped create stronger peer reviews in my online classroom.
20:32	Francesco: That’s great. I love how you structure and guide there too. And you preface it with some of those concerns. Anyone else?
20:41	Susan: Yes. This is Susan. I just want to add one thing that kind of dovetails from what the other contributors have offered up.
20:49	When I first started teaching peer review here at Excelsior, the Blackboard peer review mechanisms were kind of clunky. I had to kind of think outside the box as to how to incorporate those types of activities into the classes.
21:07	Now with Canvas, we actually have a lot more flexibility. There’s a lot of space in which collaboration can occur.
21:16	One of the things that I learned early on is that when I’m creating groups, I need to put a strong student with a not so strong student.
21:31	I had to resist the temptation to put all of the A students in one peer review group, and all of the B students in another peer review group, and that sort of thing.
21:42	My rationale for doing that is that the stronger students actually develop some teaching skills, which reinforces their own strengths, and the not as strong students benefit from the stronger students learning how to teach, if that makes sense.
22:05	Francesco: Yes. I can totally see that.
22:10	I’m really pleased to hear the variety of experiences using peer review. It sounds like there’s already a lot of activity with it at the college.
22:21	Susan, it sounds like you’ve had an easier time with Canvas. That Canvas has some other tools
22:29	Susan: Yes. I love Canvas. Then again, the classes that I teach were designed in that transitional space and so we, in a couple of key cases, designed the classes with Canvas in mind.
22:50	The classes that I’m teaching right now, I’m in a peer review with one class this week and I have one in another class next week.
22:58	Francesco: Okay. I think this gets at the second question which is challenges to using peer review.
23:05	One challenge might be the technology. We’re doing it online. That poses certain obstacles. It’s sometimes a little easier to do it in person. Especially the swapping of papers, and sitting down in groups, and so on.
23:21	Are there any other challenges that you see in using peer review?
23:33	Richard: This is Richard. We use peer review on a weekly basis in our classes in general engineering technology where the students write reflective statements on how they mastered one of the program outcomes.
23:49	The other students, there in the discussion topics for the week, pick students to provide peer review to.
23:59	We’ve got a rubric that they can use where the reflective statements we give them 10 qualities of a good reflective statement and then they’re supposed to choose at least two of those qualities to give peer review to at least two of the other students during the discussion.
24:19	The biggest challenge that I think is that some students when they find their own learning statements intimidating, they also have a problem then providing peer review to the others.
24:33	So, I think those that have weaker skills actually probably gain the most.
24:39	Some of my students are very strong; sometimes they could probably teach the class, in looking at the critiques of the statements. So, they give very valuable feedback.
24:52	When they give that feedback to their peers, generally those peers, from what I’ve seen so far, they pretty much universally take it well and are appreciative of that feedback.
25:08	I think particularly in an online class where they can’t sit and talk with each other during the breaks for the time that students have, they can do informal collaboration.
25:20	I think that the peer review assignments are very valuable for them.
25:26	Francesco: Thanks for sharing Richard.
25:29	Any other challenges that the rest of you have had or can foresee happening with peer review?
25:39	Amy: This is Amy Erickson.
25:42	The challenges I run into is when there are corresponding worksheets for the students to fill out with very specific questions similar to—like the CARES worksheet that Excelsior has—they either ignore those and don’t fill them out, so that can be a real problem even if you keep going over and over it in the instructions.
26:02	The other problem is the students aren’t qualified to… or they’ll start giving feedback in an area where they aren’t qualified. For example, they’ll say, “All of this APA formatting is incorrect.” And it’s not. It’s correct. So that’s some of the stuff I’ve run into.
26:18	Or people will insert comments that actually don’t help the paper, or they’ve not identified the thesis properly. I guess that’s an issue that I’ve encountered as far as a challenge with peer review.
26:36	Francesco: Okay. Those are some good points to think about.
26:38	Yes. I think ignoring the worksheet, obviously that makes it harder and then that feeling of not being qualified.
26:46	I think that’s going to be a challenge going through, at least until they’ve tried it once or twice and understand what their role is.
26:55	Thank you for the conversation. I think this has been a really good back and forth here.
27:00	In the next slide here, I just want to go over some best practices and some of them might speak to these challenges.
27:08	Number 1. As the instructor or the course designer, definitely build in enough time in the course or the assignment to do peer review.
27:17	It can’t be an afterthought. It really won’t work well, especially for an online course.
27:22	Make sure it is part of the design from the beginning.
27:26	Before you do peer review, model it.
27:28	Teach the skills that are required for peer review. Maybe have an activity, or a video, or something where you walk them through what peer review is and how to do it.
27:41	Present peer review as a chance to write for an audience.
27:46	So, it’s about framing the experience for them. So, they’re writing for an audience, right? I think that sort of can help to bring down the anxiety.
27:58	This one I really like. Define their role not as an evaluator, which I think opens the door to them feeling like they’re not qualified, but merely as a reader.
28:09	This, I think, speaks to the point before that this is their chance to write for an audience and the reviewer is actually a reader.
28:20	All they need to do to be qualified is to read. They just have to read it.
28:25	Whatever feedback they have—Just because they say it, doesn’t mean it’s true, and the writer will have to evaluate the feedback they get.
28:33	You know, that’s part of the peer review process.
28:36	But I think it will help to at least diffuse a lot of that anxiety around not feeling qualified.
28:45	Pairs or small groups, I’ve heard no more than groups of three, works well for peer review. Maybe they’re swapping papers, or they’re reading two papers each, or something like that.
28:58	Generally having them do peer review in large groups, or reading large amounts of papers, I think that overwhelms them.
29:07	Focus the workshops on one area at a time.
29:10	Maybe when you do peer review the first time, you focus on the argument. They’re looking at the argument.
29:15	Maybe the second time they’re looking at evidence or organization.
29:20	If its further along in the iterative process, maybe then they start looking at grammar and citation.
29:29	I really shy away from having them do that type of proof reading and looking at styling and formatting.
29:34	A lot of times they don’t have the factual knowledge for that, but they can respond to things like Was the argument persuasive; or Did this paragraph provide enough evidence to support the claim; or Is the claim clearly stated; Overall, was it well organized?
29:49	Stuff like that I find that they have a better handle on responding to and providing helpful feedback.
29:57	When it comes to grammar, citation style, and things like that, I think that’s where you really start to see that they really don’t have the expertise there to identify a sentence fragment and know how to correct it, for instance.
30:11	Provide specific guidelines. This speaks to having a rubric, maybe with specific questions. Something they can fill out.
30:19	That really helps them again to feel less stressed about the experience, more clear about what they’re supposed to do, and also, more clear about what they’re taking away from it as the writer. Now they have it all parsed out in this rubric.
30:33	Finally, provide opportunities for reflection both before and after the experience.
30:41	Ask them to just do a quick journal. Or answer a quick question. What do you want to accomplish from this? What are you struggling with in this assignment?
30:54	That’s very helpful, especially if that information gets conveyed to the peer reviewer.
30:59	I was having problems stating my thesis. Can you really look at that and help me understand whether it works or not?
31:05	That can become part of the rubric for that peer review assignment.
31:11	At the end of the experience, you can have the reviewers and the writers both respond to What was most or least helpful from this experience?
31:21	Also, come up with a plan of how they are going to revise based on this feedback. Have them sketch out their thoughts on that, a preliminary plan for revision.
31:33	So, we talked a little bit about doing peer review online.
31:37	It sounds like a few of you are using the tools that are pressed into the LMSs that you use.
31:44	That’s one way to do peer review.
31:46	Discussion boards can help with peer review. Video conferencing technology like Skype or Zoom.
31:53	You can also do peer review in the cloud now. You can share documents in Google Drive or Dropbox and then you can collectively edit content that way.
32:04	There are other Web 2.0 resources that allow you to collectively edit content.
32:10	Even face-to-face peer review can be complimented by some kind of online technology.
32:19	Peer review can be done even outside the classroom for courses that do meet in person.
32:23	Are there other technologies that you’ve used that you’d recommend for doing peer review online?
32:38	We’ll move ahead here.
32:39	So, let’s look at a couple of examples. So, one example—and I’m going to shift to my browser—one example is the CARES form which was mentioned.
32:49	This is on the Online Writing Lab. There’s actually a video that goes along with this form.
32:55	I think it’s emblematic of how most of these forms are constructed. There are some instructions up top. You put your name as the writer, and the reviewer who fills it out puts in their name.
33:06	You can see that the CARES form is actually fillable. Students can fill this out online and mail it to the instructor or to the other student. That’s a nice advantage for using our form.
33:20	It starts with actually congratulating the student on the things they did well. What are their strengths? So, start positive.
33:26	Then, start to ask clarifying questions. Which parts of the essay were confusing? What specific suggestions do you have for addressing these confusing areas?
33:36	Is there any more information that you think they need to cover in the essay? More research they need to do? More evidence they need to bring in?
33:44	Evaluate the value of the essay in terms of what specific details worked or didn’t work. What could be moved around?
33:53	Summarize. Overall, what new information have you learned or how are you thinking differently? What is the impact of this piece of writing on the reader?
34:02	It’s just a really neat way, and an easy to remember acronym for how to do peer review as a structured activity in a course.
34:11	How many of you use the CARES form? I think one person. Tell me if you use it or not. Or have used it.
34:21	Susan: I recommend it.
34:24	Francesco: Great.
34:24	Susan: I don’t require it, but I do recommend it.
34:31	Francesco: I just want to acknowledge I see that Grace also chimed in with she does peer review.
34:40	That’s one form.
34:44	We’ve linked from the presentation, we’ve linked to a couple of other forms online.
34:48	Here’s another form. It’s similarly constructed.
34:51	The writer and the reviewer put their names up top.
34:54	It’s got categories. This one is a little bit different. I actually like that this one has a column for the reviewer’s comments answering these specific questions.
35:03	But then, also a column for the writer to come back afterwards, process what the reviewer has said, and jot down some notes.
35:15	Hopefully notes that will help them to revise based on this feedback. So, that’s neat that it has that extra column there. That’s different.
35:24	Then there’s this third rubric, which has a lot more introductory content here explaining the process, the reasons behind why you think peer review is good, and so on and so forth.
35:38	Here’s a sample peer review assignment.
35:42	Again, it’s broken down by questions. This one’s not so much a worksheet for the student to fill out, but more tips for the teacher as they’re putting together a peer review rubric.
35:54	The point, of course, is that a rubric really helps the students when they’re doing the activity. It’s about good assignment design.
36:03	Creating effective assignments was the topic last time. Having a clear set of instructions will help the students through that peer review process.
36:14	All these forms are linked from this presentation, and they’ll be available on the OWL once we have this up.
36:25	I’m going to pause here before we shift to revision.
36:28	Any thoughts on how you would use peer review in your class that you’re teaching now or planning to teach? Or how you might change the way you do peer review based on what we just discussed?
36:44	Caroline: Hi. This is Caroline. I really appreciate the tools.
36:48	I’m not using it in any of my classes now, and I’ve wanted to integrate it.
36:53	I’ve used it in a traditional setting, but I haven’t figured out how to really do it well in the online environment, so I haven’t done it.
37:01	I think these tools will be helpful.
37:04	One question I’d like to ask you is do you recommend putting some meat to the participation—giving some points to you have participated in a peer review? It’s five points of your assignment.
37:20	I did that in the past. What do you think are best practices with that?
37:27	Francesco: I’ll weigh in, but I’ll also encourage others to since they’ve been doing it as well. I think it’s a good idea to give them credit for it.
37:33	Even if it’s part of the assignment, or count it as another assignment on its own.
37:40	I think it shows value. We give credit for things that we give value in the course. I would definitely give them some points for it. What do others think?
37:58	Bei: Hello Frank. This is Bei. Can you hear me?
38:01	Francesco: Yes.
38:02	Bei: Okay. So, in my class I tell students if your comment or your wrap up discussions are substantial, I will give them fully 100 to encourage them to participate actively because it’s voluntary. It’s not mandatory.
38:26	I’m thinking when they process all of the information, it does train according to the discussion rubric. It trains them in critical thinking, use of the information from references, and going deep into the content.
38:44	That’s why I try to encourage them doing the whole wrap up for the other eight or 10 students. That’s what I do.
38:57	Francesco: Great. Thanks for sharing that. Anyone else on do you grade it? Do you give points for it?
39:04	Susan: Mine are all set up as graded discussions. This is Susan.
39:09	Francesco: Okay.
39:09	Susan: In other words, it’s an activity that’s required and is lumped into the percentage that’s allocated to discussion. In my case it’s almost always 25% total. They’re basically getting, if we look at that on a 100-point scale, they’re getting about eight points, or so, give or take for participating.
39:33	It depends on the class of course. Anywhere from 8 to 15 points toward that 100.
39:43	No, I’m not calculating that correctly. Anyway, they’re getting a few points toward that 100 overall in the course.
39:52	The interesting thing about grading is that I don’t find that grading versus not grading actually improves the quality of the peer review. I just want to throw that out there.
40:02	Francesco: That’s a great point.
40:05	Caroline: That’s very interesting.
40:07	This is [unintelligible]… that you didn’t find a difference in the grading because I kind of struggled with that when I implemented it using points before saying, “Oh, should I give them credit? Should I not?” I decided to mainly because it was a class of 19-year-olds, and they probably needed credit.
40:27	Susan: Yes. The issue that I’m dealing with right now is in the class that has the active peer review is that they—I think it goes back to what we were talking about earlier with regard to not feeling as though they’re qualified to provide any sort of meaningful commentary.
40:49	The point that you were making about viewing the peer review as a reader instead of an evaluator is really important, but the assignment itself has to be set up in a way that the student can place himself or herself in that reader role realistically.
41:12	And by that I mean in the class I’ve got going right now, and I’ll summarize this very quickly, the students are peer reviewing a proposal.
41:23	It’s a choice assignment, so students can choose to write an informational report or an analytical report in the form of a proposal.
41:34	The student feasibly has to be able to place himself or herself in the role of the person who might ultimately be reading that report or proposal, as it were.
41:45	I don’t necessarily think we did a great job of setting that up in this particular class.
41:52	It’s on the docket for some critical revisions, so this particular discussion is actually quite timely.
42:01	Francesco: Okay.
42:02	Well, I’m going to move us forward here. I’d love to continue the conversation, but I want to make sure we get through the revision section too.
42:08	This is all really great. I hope we can come back to this conversation soon.
42:14	Let’s talk about revision. So, we’ve done the peer review. We’ve got the feedback now.
42:22	Someone on the call yesterday said that after the students peer review, she comes in and reviews the feedback they got and also weighs in and gives feedback.
42:32	So now they also have the teacher’s feedback. That’s up to you.
42:38	I’ve done revision where I’ve had the students peer review the draft, then the students have to revise, and then I look at it.
42:46	I’ve done it where the students peer review the draft, I review the draft, and then they do the revision.
42:53	Sometimes I’ve graded it. Sometimes I haven’t graded the draft.
42:56	Sometimes I only grade the final product, so to speak.
42:59	That’s up to you, I think.
43:02	We’re talking about revision. We’re talking about re-seeing.
43:08	We’re looking at the assignment from the 10,000 foot perspective looking down.
43:14	What we don’t mean—I think it helps to talk about what we don’t mean here—what we don’t mean is proofreading.
43:20	Proofreading is what you do when the piece of writing is ready to be published.
43:30	We’re getting it prepped. We’re cleaning it up to publish it. It’s done. It’s ready to go.
43:35	We’re going to proofread it. We’re going to edit out any lower order concerns. We call editing looking for grammar, citation, punctuation, even some sentence structure stuff. That’s the proofreading.
43:52	By revision, we mean what we call higher order concerns. What would you consider higher order concerns when reviewing a piece of writing in preparation to revise it? What are some higher order concerns?
44:15	Caroline: This is Caroline. Do the ideas flow logically?
44:20	Francesco: Yes. Great. Do the ideas flow logically. Anything else? Other higher order concerns?
44:30	Anyone else?
44:30	Susan: Are the ideas developed adequately?
44:35	Francesco: Are they developed well? Do they have enough evidence, support, and analysis?
44:43	Nicole: This is Nicole. One of the things I see sometimes is that the students make statements and then they don’t follow up with any support or any evidence to further their statement. They just leave it right there and move on to another subject.
44:58	Francesco: Yes. I’ve seen that. Great ideas here. Weak on evidence, right? There’s not enough evidence.
45:07	Caroline: Is there a thesis statement?
45:12	Francesco: That’s right. Sometimes it can be inferred or implied, but there has to be a thesis statement at some point. It has to be clear.
45:20	Attendee: You don’t get to page three and then figure out what they’re talking about.
45:24	Francesco: Right. Typically I find out by the conclusion. Oh. That’s the argument. I see it now.
45:35	These are all what we call higher order concerns that we look for when we revise.
45:44	We’re going to get to a list of those concerns in a sec.
45:48	I think we revise because we know that writing is an iterative process. It’s a process over time.
45:56	Writing is done when we say it is, not because it’s done. We’re just done with it at that point.
46:02	Writing is a process of discovery, and revision can occur at any stage of that process.
46:10	Whether we’re prewriting, we’re drafting, we’re researching, we’re revising, we’re responding to each other’s writing, revision is happening at all those stages. We’re re-seeing, rethinking the writing.
46:28	I also want to stress that we revise all the time.
46:33	I think this is a good point to make with our students. Revision is just part of life. It’s part of how we work.
46:40	We get feedback from people on the job and that helps us to revise.
46:45	We write memos and emails, and then we reread them and revise them before we send them out.
46:49	We revise our syllabi all the time. We revise course assignments from semester to semester.
46:55	We revise social media posts. They revise social media posts.
47:01	If you write a blog, you’re revising it.
47:03	If you are composing video or image content, like Snapchat, is really revision. You’re taking image content. You’re revising it, putting it through some filters, and then posting it. So, that’s revision.
47:16	On the right there, I decided to share an example of how we revised the content for these presentations.
47:25	There was a slide in the first presentation that got us into defining WAC, WID, and WTL.
47:32	The top image is the image we started with, and the bottom image is what we ended up publishing actually and presenting to you.
47:39	So, that went through an iterative process of revision.
47:43	Revision is just a natural and normal process for composing. Whether it’s composing writing, or composing multimedia, it’s part of the process.
47:51	There are different revision strategies.
47:55	I suggest including drafting in the assignment timeline. Make time for it, just like with peer review. Make time for revision.
48:03	Decide how you’re going to grade it or what kind of value you’re going to give it but give it some value.
48:08	Incorporate peer review so they’re getting new ideas to help them revise.
48:13	Decide whether to read or grade the drafts. That’s up to you.
48:17	Guide students in identifying their weaknesses with a rubric.
48:21	Advise the students to read their draft out loud. That really speaks to the proofreading part of it, but it also helps them to hear what sometimes the mind doesn’t see.
48:30	Explain the difference between higher order and lower order concerns.
48:34	We started to talk about that, and I think we have a pretty good handle on that here.
48:39	When we revise, we’re focusing on those higher order concerns first.
48:43	It’s not that we don’t focus on the lower order at all, but we first want to address the higher order concerns. The things that you talked about.
48:50	Thesis.
48:52	Does it address the purpose of the assignment?
48:57	Is it written for the right audience?
48:58	Is there a logical flow and organization?
49:01	Is there any missing information that should be brought in—more research, more support?
49:06	Is there anything redundant that could be cut out? Sometimes they sort of go off on a tangent or a digression.
49:12	Can they enhance the ideas somehow? Do they need it enhance it somehow?
49:17	The lower order, as we mentioned, are the spelling, the grammar, punctuation, citation, style, sentence structure, word choice.
49:25	These things are just really polishing the text once it’s ready to go.
49:30	We want to go through those higher order first.
49:34	When we’re talking about tips on revision, we always say start with the higher order concerns. Trust these first.
49:42	Some other tips for the higher order concerns. Walk away from the document. Get a fresh perspective. Come back. Work on it again.
49:52	Read through the document with the higher order concerns in mind.
49:55	Maybe have a rubric. Look for thesis. Look for organization. Look for transition. For whatever it is.
50:03	Just be conscious of the fact that you’re looking at the document for those things.
50:08	Use a heuristic, a rubric.
50:10	Consider feedback from other readers. That always helps to get that new fresh perspective. Right? Have other people read it.
50:17	We do that when we submit journal articles or grant proposals or whatever. We get feedback from our peers, and we revise.
50:26	Make sure it follows the assignment. Go back and look at that assignment again. Did you answer all the questions?
50:31	I often find myself telling my students, “Your essay answered part of the question, but not all of it.
50:37	The lower order concerns have tips here too.
50:39	Address these later after the higher order.
50:42	It helps to have a printed copy to review with a pencil or pen and sort of mark up the text as you’re going through.
50:49	Find the errors.
50:50	Read the document out loud, especially the year errors.
50:54	Use a grammar check. That makes it easy.
50:56	Have someone else read it.
50:59	Then, just focus on a few things at a time. It can become overwhelming.
51:02	We know as a teacher if we mark up a text with too much red ink, we’re sure to paralyze that student.
51:09	Just focus on the top three things that they can correct, as far as lower order concerns.
51:20	Some folks like to grade or develop assignments with portfolios. I just wanted to touch on this for a moment here. How many people here use portfolios?
51:33	Susan: I do.
51:34	Francesco: Okay.
51:35	Bei: Yes. We do too.
51:37	Francesco: Okay. Alright. I know we at the college we now have Portfolium. So, we do have technology to assist with using portfolios.
51:46	It’s a different process. It’s a holistic process for grading student work.
51:51	It also helps the student because they get to curate what goes into that portfolio.
51:57	They can revise that work.
52:00	Reflection can be part of that process. They can include reflective pieces in that portfolio.
52:06	Then they’ve got this set of work that they can share. Just like an art portfolio. They can say, “this is my opus; this is what I’ve produced.”
52:13	They can share with not only their teachers and their peers, but also with employers.
52:19	And really show, especially if they’re writing in the disciplines, and they can show the ways that they are competent in that discipline.
52:27	There are different portfolio platforms out there that can help with doing portfolios.
52:37	You can see how the iterative process through which these writing assignments unfolded in that portfolio.
52:44	Digication. Portfolium. Mahara. WordPress. Using blog sites. Google Sites. LMSs also sometimes have portfolio tools.
52:53	These are different digital tools that will help with assignments that are multistage. That have a revision process.
53:01	You can show that revision in a portfolio as the assignment has grown over time. As that product has grown through this iterative process.
53:15	We’ve come to the end here.
53:17	I just want to inform you that we will have a fourth and final presentation which will be on effective feedback and using rubrics.
53:27	That one will be coming up, probably in March. Please stay tuned for that. We will be sending out an announcement soon and get that scheduled.
53:35	I want to thank you for your time today.
53:37	Thank you for joining us and for the excellent commentary and feedback and discussion we’ve had. I think it’s made it really worthwhile for me and I hope we can continue the discussion.
53:48	Thanks again.