Observing Online Courses...

What resources do your faculty use to “observe” instructors and their online courses, as they would observe a F2F instructor and course? Often these observations are a regular part of a program review or one’s P&T process. Some faculty feel they don’t know how to observe an online course as they haven’t taken or taught one. I’ve found a smattering of rubrics at other institutions, but would like to get feedback from this group.

Thanks,

Scott

Scott Robison, Ph.D.
Director, Learning Technologies and Online Education
105 Lamson Learning Commons, MSC #47B
Plymouth State University
Plymouth, NH  03264
603.535.2262

Thanks Molly,

Yes, I am interested in resources for observing the teaching of the online course, not the course itself. I understand these can’t be completely separated. Imagine two sections of a well-designed online course are taught by different instructors; one instructor gets shining student course evaluations and the other does not. It wasn’t the course that changed. It could have been the students. It probably was the instructor. What rubrics are people using to help administrators identify the differences between the two instructors’ interactions with students and with the content?

From: The EDUCAUSE Blended and Online Learning Constituent Group Listserv [mailto:BLEND-ONLINE@LISTSERV.EDUCAUSE.EDU] On Behalf Of Molly Baker
Sent: Tuesday, October 30, 2012 4:08 PM
To: BLEND-ONLINE@LISTSERV.EDUCAUSE.EDU
Subject: Re: [BLEND-ONLINE] Observing Online Courses...

There are several widely-accepted rubrics (Quality Matters, the ION one in Illinois, etc.), but in my opinion, they focus on course design, not on teaching the course. When I was at Black Hawk College, we created a Best Practices for Exemplary Online Teaching set of standards based on the Chickering and Gamson's "7 Principles of Good Practice for Undergraduate Education" meta-analysis. Individual best practices for online teaching were pulled from the literature and listed as possibilities under each of the 7 principles, and an 8th was added with some of the course design elements not already mentioned in the first 7. In other words, we created a local document that could assist faculty in doing self-assessment, peer evaluations of each other's courses, and potentially institutional review of online courses. However, our instrument was not used for institutional assessment because it was not approved as part of the faculty [union] contract. It is important for a document like this to be shared with the faculty ahead of time so that they know how their courses are going to be evaluated. I also think it is helpful to have several people evaluate various aspects of online courses, such as someone who is an expert in online education who can evaluate the learning experiences and course design elements of the course, someone from the faculty member's department who can evaluate the quality and accuracy of the course content, as well as the administrator whose job it is to evaluate teaching. If the institution uses a type of rubric or assessment document when evaluating face-to-face teaching, it needs to be vetted by online experts to determine if it emphasizes appropriate, comparable variables in the online environment. For example, if activities to promote student engagement is on that form...what does that look like online? Not all administrators or faculty who have not taught online would know what to look for as indicators of student engagement.

Hello Scott,

I’m not sure if this listserv allows attachments, but if so, you’ll find attached the review form we use at Ottawa University. It probably doesn’t match your needs exactly, but might provide some basis for further discussion at your institution.  As the document name implies, these reviews are for new instructors who are seeking certification for online teaching and the reviews are completed by their peers.

Best,

Carine Ullom

Thanks Carine. We don’t have a peer review component in our online certification process (our department reviews the course before and after the course is offered), but the review form you sent is along the lines of what I am putting together for department chairs/administrators who need to observe online instructors. I appreciate hearing what you are using. I’m putting together an online course for “observers” with the primary goal of showing people where they will find evidence of, for example, “Demonstrates enthusiasm in written form (active, energetic, expressive).” I think people are aware of what a good (F2F) course is, but finding the evidence in an online environment is challenging for some.

One question, what specifically are faculty assessing/looking for regarding item #3?

-Scott

We originally started with a course evaluation rubric that evaluated the course prior to launch.  But we have always believed that this is only half of the picture because what really makes a course successful (whether online or in the classroom) is what happens when the course is live.   We are beginning to work on a rubric that can be used early on in the course or midterm and another that could be used at the end of course.  All of these rubrics can be used by the faculty to self-evaluate.  They will be made available to them on the faculty resource center.  It will help them understand the expected basic standards especially if they are new to online teaching. 

But these rubrics are also being designed to be used by evaluators/observers.  In this way, we can create a common language around the criteria and the faculty will already be familiar with the rubrics.

Molly,

I’m familiar with the “7 Principles…” and believe F2F and online courses should be held to very comparable standards. The trick is to educate (some) people on where to find the evidence in online courses. I’m putting together an online course for “observers” with the primary goal of showing people where/how they might find such evidence.

Thanks,

Scott

From: The EDUCAUSE Blended and Online Learning Constituent Group Listserv [mailto:BLEND-ONLINE@LISTSERV.EDUCAUSE.EDU] On Behalf Of Molly Baker
Sent: Tuesday, October 30, 2012 4:58 PM
To: BLEND-ONLINE@LISTSERV.EDUCAUSE.EDU
Subject: Re: [BLEND-ONLINE] Observing Online Courses...

Scott

Please, I’d love to see Phase 1 and 2 also.