Case Study 4: Yakima Valley Community College: Using Near-Real-Time Data to Increase Student Success

Published:: Wednesday, May 2, 2012
: Books and Monographs

Author(s) and Contributors:: Author(s): Wilma Dulin Shelia Delquadri Nicole Melander

Source(s) and Collection(s):: Sources(s): EDUCAUSE Publications

ParentTopics:: Data Administration and Management Data Governance Student Success Learning Analytics

Abstract

Wilma Dulin, Sheila Delquadri, and Nicole M. Melander

Overview

In July 2006, Yakima Valley Community College (YVCC) joined the Achieving the Dream national reform network and began the process of transformation. Over the next four years, YVCC moved from an institution with very limited research capability to one with an Office of Institutional Effectiveness (OIE) dedicated to gathering and analyzing data on student outcomes—and to putting that information in the hands of faculty members.

Initially, YVCC's data analysis focused on course pass rates and sequence completion in English and math. Students who lack proficiency in basic English and math often have difficulty in a wide variety of other courses, and this affects success throughout the college. As the college began to ask questions about success, common data elements needed by all departments were identified. The OIE created a web-based data analysis tool with pivot table functionality that enabled YVCC personnel to access data related to placement, enrollment, and course completion. The data are disaggregated by ethnicity, gender, location, and mode of instruction. Faculty and their department heads use these data to analyze curricular areas, to strategize activities to increase student success, and to help monitor their own progress toward accreditation goals. Data are updated regularly, providing near-real-time information.

Because the data pulled was consistently provided from replicable sources, confidence in the accuracy of this data has increased. Now, data is required before YVCC invests in student-success strategies, including the development of course schedules that are based on course-taking patterns and success rather than just on past enrollments.

Examples

Data on student success have impacted faculty decision making and enrollment management in three different areas at YVCC.

Placement Data Available to Advisors

In 2006–07, YVCC analyzed data on first-quarter and first-year retention and conducted focus groups with students, faculty, staff, and community members on "barriers to success." The most frequently cited barrier was the lack of clear academic guidance.

In spring 2008, YVCC created a new intake process that includes an online orientation to placement testing and a two-hour mandatory "New Student Orientation and Registration" session. Each summer before orientation, the registrar provides the (faculty) advising team with placement data on incoming students. The advisors use the data to determine whether courses available are appropriate given the students' achievement levels. Courses are added or eliminated from the schedule accordingly.

Completion Data Available to Faculty

In any given year at YVCC, more than 50 percent of entering students are required to take developmental English, and more than 85 percent are required to take one or more levels of developmental math. An OIE analysis revealed that students who begin at the lowest levels of math have very low rates of earning quantitative course credits required for degrees. YVCC conducted focus groups with developmental students to redesign developmental courses, and three significant changes were made:

The Math Department created four different pathways to courses meeting the quantitative degree requirement, and it also created a new course to help students transition directly into a math pathway—if students do not have the skills to directly enter one of these four pathways, they can take this new course that will enable them to enter one of the four pathways.
Faculty members analyzed completion rates in precollege course sequences and determined that proper placement required more than scores from the COMPASS English test. They asked for a writing sample and basic skills tests (Comprehensive Adult Student Assessment Systems) from incoming students to assess proficiency in reading, math, listening, speaking, and writing in order to place students in one of three levels of English coursework.
Student Support Services (SSS) was redesigned in the fall 2010 so that entering students were required to attend a new SSS Student Orientation session and to enroll in their first quarter in an SSS Learning Community.

Correlated Enrollment and Placement Data Available to Faculty

The OIE has conducted numerous analyses of the effectiveness of placement cut scores and prerequisites on student success. For example, a detailed analysis of enrollment data was matched with COMPASS placement data in English and math. It was discovered that success in psychology was significantly correlated with placement into English 101 and Intermediate Algebra. Based on this, a decision was made to add these as prerequisites for Psychology 100.

Results

The results of the strategies described above are summarized in Table 1.

Table 1. Results of the Three Strategies

Strategy	Outcome	Measures	Supporting Data
ADVISING New-Student Orientation and Advising	More students enroll in the courses they need and courses they can complete; higher student satisfaction and retention.	Tracked via "Incoming Student Survey" and "Annual Fall Student Survey"; tracked quantitatively via first-quarter and first-year retention.	More than 90 percent of new students reported positive perceptions of the intake process; first-quarter and first-year retention rates both increased 4 percent.
SCHEDULING Precollege Course Design and Management	Significantly more students enroll in precollege math. These students more smoothly transition to credit-bearing courses.	Precollege math enrollment levels.	YVCC hired two additional math instructors in 2010–11 due to increased enrollment.
	More accurate placement for students requiring developmental English.	Placement levels and course completion for precollege English.	20 percent of students place at a higher level of English without a decrease in student success.
	Higher retention of SSS students participating in learning communities.	Enrollment of SSS students in math.	80 percent of first-quarter SSS students enrolled in math compared to 58 percent of matched cohort.
SCHEDULING Placement and Prerequisites	More students succeeding in courses and fewer dropping courses; smoother course enrollment patterns.	Course grades Course drop rates Faculty satisfaction	Completion of "C or better" improved from 64 percent to 75 percent, and course drop rates were cut from 16 percent to 7 percent in Psychology 100.

Challenges

YVCC faced a number of challenges. The following have recurred in other initiatives related to the shift to an evidence-based culture.

Faculty Are Not Data Analysts: Faculty members are experts in their fields of study, yet they have only limited ability to analyze data or understand the difficulty of retrieving information in particular ways. It takes time for faculty members to learn how data are gathered, stored, and reported.
Transparency Is Threatening: Faculty worry that data will be used for their own evaluation or that certain faculty will be singled out based on results. Policies and procedures are needed to address these concerns.
A Single Version of the Truth: Data on the same topic (e.g., the same question) may vary from one data source to another due to the timing of the data extract, the cohort of students captured in a particular data source, and/or the phrasing of a research question. Protocols and documentation address frustration with these data challenges.

Lessons Learned

In the course of this process, YVCC learned a great deal and came away with the following main lessons:

Determine Intent: Understand the intended use of the data by the faculty member. Communicate data sources, meanings, and limitations. Spend time up front understanding the data requested. Understand what the requestor anticipates the data will reveal.
Document Data Sources: Document data sources, including the actual "pathway" to the data. Archive reports for "look back" capabilities.
Develop Data-Governance Policies: Develop data-access policies and notification protocols. Define data ownership—particularly course-level data owned by faculty.

Wilma Dulin has been a faculty member at YVCC for twenty-three years. In addition to teaching student development courses aimed at underprepared first-generation college students, she directs the Office of Institutional Effectiveness and serves as the Title V Activity II Diversity co-coordinator. She was co-leader of the initial YVCC Core and Data teams and now convenes the Institutional Effectiveness Team. Sheila Delquadri has eight years of experience as a data coordinator for a variety of programs, including GEAR UP, Title V, ATD, and other student-success programs. She serves as the research analyst for the Office of Institutional Effectiveness. She was an original member of the Core and Data teams and continues to oversee research design and data collection. Nicole M. Melander is Chief Technology Officer (CTO) for Achieving the Dream, Inc. (ATD). In this role, she ensures programs are informed and supported by cutting-edge technology. Melander previously worked at Oracle Corporation, Microsoft, and Deloitte Consulting, where she acquired in-depth knowledge of social media, educational technologies, e-business software, and large database systems.

Download Resources

Download PDF