Implementation Guide for the ELI Student/Faculty Questionnaire

Introduction

The ELI Student/Faculty Questionnaire was designed to help your institution assess student expectations of technology-enhanced learning. The faculty version helps you explore faculty experiences and preferences for learning technologies, as well as faculty perceptions about student learning. The two versions capture the same concepts, but from student and faculty viewpoints. The questionnaire allows institutions to compare faculty and student responses, which may reveal a gap between student expectations and faculty delivery. It also allows you to examine the gap between how your students like to learn and how your faculty think they learn. These results can then catalyze discussions that allow faculty, administrators, and students to discuss differences in expectations.

This guide is not designed as a tutorial on survey research methodology. We strongly recommend including on your implementation team at least one individual who is well versed in survey methodology. This guide contains general guidelines and strategies for dealing with common problems that, left unchecked, can weaken the validity of your findings from administering the questionnaire.

Administering the Questionnaire: Sampling

Your primary goal is not to have a large number of questionnaires returned; your goal is to gather questionnaires that are representative of your student and faculty populations.

Most surveys use either random samples or population samples. The key element for drawing a random sample of students and faculty is to ensure that every member of that population has an equal chance of being included in the sample. You may have a smaller number of respondents but, assuming a high response rate, they will still be representative of your population. A drawback to this approach is that respondents may feel "targeted" because random sampling necessitates follow-up messages urging questionnaire submission.

Because most higher education institutions now have e-mail mailing lists that include all faculty and students, researchers are using population sampling more frequently. Population sampling differs from random sampling in that the entire target population is asked to respond to the questionnaire. This method is fully inclusive and avoids targeting, but the issue of response rate can present a significant challenge.

Administering the Questionnaire: Response Rate

Since the ultimate purpose of the ELI Student/Faculty Questionnaire is to promote activities that narrow the gap between student expectations and faculty delivery, the data collected must be representative of both groups. The problem of low response rate, therefore, must be confronted.

A difficult problem arises in these processes if the reason people respond (or do not respond) is related to your questionnaire's content. If this occurs, your data could be biased. In questionnaires that try to assess an interviewee's technological aptitude, those who are more comfortable using technology may be more likely to respond. This bias will present an inaccurate representation of student and faculty perceptions and abilities.

The design of the instrument has a direct effect on response rates. The ELI Student/Faculty Questionnaire was designed to be brief and to be easily understood. Each of the student and faculty sections of the questionnaire takes an average of eight minutes to complete.

The following will have a positive impact on response rates:

• Have the questionnaire endorsed by a senior college or university officer before it is deployed.
• Do a prenotification. Your message, in essence, should be "An important questionnaire is coming and it will require very little of your time."
• Follow-up—the first reminder should come no more than one week after the questionnaire is deployed.
• Remind respondents that their participation is important to the institution. Make sure they understand that response rates are tied to accuracy.
• Communicate this at the prenotification stage, the deployment stage, and in your follow-up communications.
• Ensure potential respondents of confidentiality.

Questionnaire Structure

The ELI Student/Faculty Questionnaire has four parts:

• Part I (Q 1–5) examines student/faculty use of e-mail and selected learning technologies.
• Part II (Q 6) of the student version asks what learning technologies students either expect or would like to see enhance their learning. The faculty version asks what learning technologies faculty feel comfortable using, or would like to use, to enhance their teaching.
• Part III (Q 7) presents several learner activities and attitudes. This section can be used to discover how your students perceive themselves as learners and to see if faculty share this perception. The faculty tool also includes several questions about your institution's support and training capacities. The student version does not contain these questions.
• Part IV contains background variables that allow you to segment the data for deeper analyses.

What to Do with the Results

If you were successful in generating interest on your campus, your respondents will be impatient for feedback. If the lag between administering the questionnaire and providing the results is too long, you will lose the momentum generated by the process.

The ELI Student/Faculty Questionnaire results are not the end of the process; they should be seen as the beginning. Start with a campus-wide distribution of the results using communication channels such as internal Web postings, the student newspaper, and faculty/staff newsletters. In these communications, concentrate on a discussion that centers on the size of the two gaps at your institution. If the results show a significant distance between students and faculty in these areas, publicizing the findings can catalyze an important campus dialogue. We recommend that these dialogues take the form of student/faculty focus groups initiated at a subunit (department/college) level. For suggestions on how to use focus groups to continue the momentum started by the process, see the ELI guide, "Conducting Focus Groups as a Follow-Up to the ELI Student/Faculty Questionnaire."