Main Nav

Analytics that Inform the University

In just one week the ELI 2012 Online Spring Focus Session will begin. One of the presenters, Thomas B. Cavanagh, assistant vice president of distributed learning at the University of Central Florida will present on how analytics can go beyond course development to inform at the instutional level. Below is an excerpt from an upcoming article Mr. Cavanagh is finalizing for JALN, and it will be the focus of his session on Wednesday, April 11, followed by a brief Q&A.

Analytics that Inform the University

Much has been written about the potential of learning analytics at the course level. Certainly, there is demonstrable value in being able to identify at risk students and proactively intervene to get them back on track. Likewise, mining through the usage of instructional tools to understand effective technology-based teaching strategies can yield important trends that can inform future course development.

However, the same potential exists to leverage data analytics strategically at the institutional level. Being able to examine macro data across departments, colleges, and the larger university can reveal institutional opportunities that might have otherwise remained hidden.

At the University of Central Florida (UCF), the Center for Distributed Learning (CDL) is responsible for overseeing this institutional lookout of what is a combination of what Barneveld, Arnold, and Campbell have called “Business Analytics” and “Action Analytics.” To do this, we maintain simultaneous “top-down” and “bottom-up” views of what is happening across the university related to distributed learning (completely online, blended, and lecture-capture courses and programs).

From a top-down perspective, CDL has developed a proprietary data mining platform called the Executive Information System (EIS). The EIS began as a skunkworks project to better automate CDL’s ability to answer various questions from senior administration. Over time it has grown into an indispensable tool in the management of a high-growth online learning initiative at the second-largest university in the nation.

Among the diverse set of functions the EIS offers are:

  • manages faculty development scheduling and credentialing to teach online.
  • maintains historical faculty teaching records across all modalities, as well as master course schedule data.
  • tracks productivity data (e.g., registrations, sections, student credit hours, etc.) by campus, college, and modality.
  • permits program tracking for regional accreditation and state governing board reporting.
  • monitors student demographics. 

While the EIS is a powerful suite of features, it is constantly evolving, adding reports, creating a new question for every question it answers. Perhaps its most powerful aspect is the fact that a majority of the data that it analyzes and reports on exist in various other locations throughout the university (such as Institutional Research). However, the EIS aggregates these existing data with some manually-entered data to create a robust architecture that allows UCF to maintain a top-down view of what is happening with technology-based learning at all levels across the entire institution.

From a bottom-up point of view, CDL’s Research Initiative for Teaching Effectiveness (RITE) maintains a robust program of continual analysis and interpretation of data points such as student success, withdrawal, and perception of instruction (end of course evaluations). If the EIS top-down data are used to scan the university’s distributed learning initiative from a primarily quantitative standpoint, the RITE bottom-up data are used to identify trends, compare performance, and track the qualitative progress of distributed learning.

These bottom-up student performance and perception data also help to inform decision-making at all levels of the university. New inquiries by RITE researchers have focused recently on grade point average (GPA) as a more reliable predictor of student success than other typical variables that are often studied in the context of learning analytics.

Again, like the top-down data being leveraged by the EIS, RITE’s data already exist in different university locations. The team is able to collect these disparate data points and combine them with original data (such as direct surveys) to produce actionable research.

In all of UCF’s analytical efforts is the theme of trying to ensure that the results are actionable. For without being able to do something productive with the data, the collection and reporting of information becomes an abstract art, of little use to the institution or students. So, for example, within the EIS’s program tracking is the ability to proactively report on every degree in the university and determine how close each is to being 100% available online. CDL leadership can use those reports when meeting with colleges and departments to shape strategic conversations about how to allocate resources to exploit opportunities that may not have been previously known without the data. Is a particular program only two courses away from being available completely online, thus making it “low hanging fruit” for online development? Does it make strategic and pedagogical sense to prioritize resources to develop those two courses and place the entire program online?

Likewise, understanding which bottom-up predictors, such as GPA, can be leveraged to impact student performance via early intervention or special advising or extra tutoring can make all the difference in the world to both the individual student and, when writ large across the university, for the entire institution.