Main Nav

Message from shelf@westernu.edu

Hello All,

 

We are currently considering rolling out an online, high stakes solution for our osteopathic medical school.

 

Would anyone be willing, in this forum, or a private phone call, to share their thoughts and experiences regarding, e.g. lessons learned for vision, planning, budgeting, vendor selection, governance, logistics, gotchas, the good, the bad, the ugly, etc.?

 

As always, the hard won knowledge and wisdom of this wonderful group are infinitely appreciated, and thank you in advance for sharing.

 

Sincerely,

 

Scott Helf, DO, MSIT

Chief Technology Officer-COMP

Director, Academic Informatics

Assistant Professor

 

Department of Academic Informatics

Office of Academic Affairs

College of Osteopathic Medicine of the Pacific

Western University of Health Sciences

309 East 2nd Street

Pomona, CA  91766

 

909-781-4353

shelf@westernu.edu

 

www.westernu.edu

 

********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.

Comments

Message from luke.fernandez@gmail.com

Scott, We've coded our own (chitester.weber.edu). Chi is actually commercially available (although we don't do any marketing and are hoping to convert it to an open source license in a bit). Chi delivers over 100,000 online high stakes tests a semester. It caters in large part to courses in our College of Health Professions. I'd be happy to answer some questions about markets, technical and pedagogical needs etc. Cheers, Luke Fernandez, PhD Manager of Program and Technology Development Weber State University
Scott, I assume by "high stakes" you mean "a student cannot matriculate until the exam is passed at an acceptable level". The College of Business and Economics at California State University, Northridge has been running a suite of "high stakes" exams for 10 years now. This exam consists of six individual exams--each one a set of multiple choice questions only (no open-ended questions or qualitative evaluation). If a student cannot pass an exam in two different semesters, the student *cannot* be a College of Business student (our entire campus is ~36,000 headcount; our College is ~6,000 headcount). We've given 7500 exams (700 students times 6 exams plus many take one or more exams more than once) or more a semester plus another couple of hundred each Summer. We've had a small issue with server-side performance, but nothing major that our IT folks can't help with (just have your IT admins. plan for RDBMS write-concurrency, especially near the end of the semester...sigh). We initially used WebCT (institutionally-managed and in-sourced); we now use Moodle (institutionally-managed and out-sourced). Both environments have worked fine (largely due to some preliminary design and coordination by IT instructional designers and instructor-of-record faculty). Initially, teams of faculty authors used ExamView on the desktop to generate the six exam banks; now we just edit the questions and answers directly in Moodle. For us, each exam is unique in that each exam is programmed to choose 16 random questions (and random by level of difficulty) from a bank of 200 questions in the exam bank (there are six exam banks, one for each of the six exams--accounting, law, statistics, etc...). Moodle provides the exam results immediately, and the students can look up their current exam status in Moodle as well. We also included "where to find review material on the our intranet to study this topic" in the Answer response to help the students pass the exam on a retake (if needed). As for time, Moodle is configured for 35 minutes per exam. We limit the number of attempts in Moodle to 3, and we set Moodle to require at least one day between exams (but we change this manually as we move through the semester). As for security, we configured Moodle to limit the exams to a very narrow set of on-campus, wireline (not VPN or wireless) Class B subnets. We set up a special-purpose, small (28 seat) lab just for this test taking process. As for physical security, we just follow general Sylvan Learning Center standards--students show a campus ID, we have security cameras, we have locked-down, single purpose PC's (this takes a little work, but can be done with good IT/desktop admin. coordination/support) in this dedicated lab, and we also have the students take everything out of their pockets. It's likely that no security is perfect, but we are getting better. Moodle 2.0 changes the quiz module a bit, so you'll need to deal with that (we are on 1.9.x). On the plus side, it's relatively easy to "import" questions into a Moodle test bank (but double-check graphs, equations, and associated filenames, etc.) Also, I have yet to see a good solution for off-campus "high stakes" exam test taking. Yes, I know some browser "lock-down" applications exist, but at least our College isn't comfortable with the reliability and validity of such approaches. Others on this list might disagree. My guess is that there now exists Higher Ed. segment IT service providers to handle nearly 100% of what you need (heck--you could probably just rent space from Sylvan Learning or similar and be done with it on the infrastructure side). Hope this helps. And I'm relatively local...I'd be happy to chat on the phone (818-677-4524) or drive out to the Pomona area to chat about this stuff as needed (I did my graduate work at Claremont Graduate University). Best, Wayne Smith, Ph.D. Department of Management California State University, Northridge Educause Leadership Institute (class of 1999) On 10/28/2011 3:59 PM, Scott Helf wrote: > Hello All, > > We are currently considering rolling out an online, high stakes solution > for our osteopathic medical school. > > Would anyone be willing, in this forum, or a private phone call, to > share their thoughts and experiences regarding, e.g. lessons learned for > vision, planning, budgeting, vendor selection, governance, logistics, > gotchas, the good, the bad, the ugly, etc.? > > As always, the hard won knowledge and wisdom of this wonderful group are > infinitely appreciated, and thank you in advance for sharing. > > Sincerely, > > Scott Helf, DO, MSIT > > Chief Technology Officer-COMP > > Director, Academic Informatics > > Assistant Professor > > Department of Academic Informatics > > Office of Academic Affairs > > College of Osteopathic Medicine of the Pacific > > Western University of Health Sciences > > 309 East 2nd Street > > Pomona, CA 91766 > > 909-781-4353 > > shelf@westernu.edu > > www.westernu.edu > > ********** Participation and subscription information for this EDUCAUSE > Constituent Group discussion list can be found at > http://www.educause.edu/groups/. > ********** Participation and subscription information for this EDUCAUSE Constituent Group discussion list can be found at http://www.educause.edu/groups/.
Close
Close


Annual Conference
September 29–October 2
Register Now!

Events for all Levels and Interests

Whether you're looking for a conference to attend face-to-face to connect with peers, or for an online event for team professional development, see what's upcoming.

Close

Digital Badges
Member recognition effort
Earn yours >

Career Center


Leadership and Management Programs

EDUCAUSE Institute
Project Management

 

 

Jump Start Your Career Growth

Explore EDUCAUSE professional development opportunities that match your career aspirations and desired level of time investment through our interactive online guide.

 

Close
EDUCAUSE organizes its efforts around three IT Focus Areas

 

 

Join These Programs If Your Focus Is

Close

Get on the Higher Ed IT Map

Employees of EDUCAUSE member institutions and organizations are invited to create individual profiles.
 

 

Close

2014 Strategic Priorities

  • Building the Profession
  • IT as a Game Changer
  • Foundations


Learn More >

Uncommon Thinking for the Common Good™

EDUCAUSE is the foremost community of higher education IT leaders and professionals.