University leaders and faculty are responsible for the health of their academic program portfolio – the heart of the institution – but often lack the diagnostic tools and strategies to improve its fitness. What are the right data sources? What analysis should be done? Who should be involved? How often? To answer these questions, Gray offers a free, comprehensive, four-part Master Class. The class is results from years of data analysis and hundreds of workshops with institutions of all sizes and types, from rural community colleges to large, Tier 1 research institutions.
The Master Class describes data and decision intelligence software. It lays out the processes that higher-education decision-makers need to support and empower data-informed, not data-driven, decisions. As we will discuss, effective program evaluation and management requires data and the institutional knowledge and informed judgment of faculty, administrators, and institutional researchers.
The Four Cornerstones of a Program Evaluation System
Our first Master Class session will start with an overview of the four cornerstones of a program evaluation system: mission, academics, markets, and margins. Next, we explain how to distinguish mission-critical programs that may merit special treatment from those that should stand on their own. After mission, we will dive into market demand: how to gather and analyze the best data on student demand, employment, competition, and degree fit. We will share sources for timely, accurate data on student demand by program and market. We will describe how to gather and analyze competitive data so you can distinguish competitive markets from saturated markets. We will debunk myths about the links between jobs and academic programs and explain how to get sound data on the jobs, wages, and skill requirements for each of your majors. We will close the analysis of markets by describing how to determine the appropriate degree levels for a program.
The Importance of Program Economics
Too often, we see people focusing on one or more dimensions of program evaluation but not all of them. During our second Master Class session, participants will learn why program economics are necessary. Analyzing program economics helps avoid cutting small programs that make money or expensive ones that have high margins. We will explain how to use data on program revenue, cost, and margin to evaluate programs and improve curricular efficiency.
Integrating Student Success into Decision-Making
Markets and Economics are important components of a program evaluation system, but your institutional mission is likely about students. Our third session will focus on integrating data on student demographics and academic performance to inform decision-making. Reviewing reports you can use to assess failure rates, retention rates, and graduation rates by program, course, ethnicity, and gender can make this process easier, allowing you to see how you can use this data to incorporate diversity and equity in your program evaluations. This Master Class session focuses on this process and the steps needed to pinpoint strengths and weaknesses in the delivery of your programs.
Predicting Success with AI and Machine Learning
Did you know that it is now possible to predict the number of students in a new program and its likely revenue, cost, and margin? With this information, you can gauge if a program has reached its full market potential or has room to grow. Understanding the future of higher education and program evaluation tools and techniques is necessary, now more than ever, to stay one step ahead. That is why our fourth and final Master Class session in the series touches on the implications of emerging artificial intelligence platforms like ChatGPT.
For complete program evaluation, institutions need to focus on program evaluation processes, people, and management. Importantly, program evaluation is not data-informed; it is data-informed and relies on the institutional knowledge of faculty, administrators, and institutional researchers to determine what programs to stop, start, sustain, or grow. Effective program evaluation ensures that your program portfolio is up and running and stays lean, strong, flexible, and ready for the competition. Next time someone asks if you know where your program evaluation system is, there will be no panic involved.
