Putting Institutional Performance in Perspective

Marcus SzymanoskiPerspectives - Masthead  

By Marcus Szymanoski  

I doubt you would be impressed if I told you it takes me 25 minutes to run one mile. After all, most healthy adults can walk a mile in less time than that. If I told you this particular mile is the Manitou Incline, which ascends 2,000 feet up a Colorado Rocky mountain, you might be more impressed. The difference is context.   

Context is critically important in many aspects of life, and postsecondary education outcomes are certainly no exception. As policymakers increasingly look to measurements of institutional outcomes to allocate financial aid funds and influence behavior, many within the financial aid community have expressed valid concerns about the potential unintended consequences. A key U.S. Department of Education (ED) official acknowledged these concerns during a recent congressional briefing. To strike a balance between the goals of enabling access and improving higher education attainment, institutions should not be discouraged from serving the very students necessary to achieve these goals. 

One way to address this challenge is to evaluate institutional metrics in context with predicted outcomes that reflect the characteristics of the student populations being served. While institutional practices certainly have an impact on student success, other factors profoundly influence that success as well. It is important to recognize these factors and account for them. Numerous studies, including work by Tom Mortenson at The Pell Institute for the Study of Opportunity in Higher Education and the American Institutes for Research, have demonstrated that factors such as socioeconomic status are predictive of student outcomes. In addition to income, which has been established repeatedly as a primary factor in students’ educational persistence and success, ED has identified risk attributes of students that correlate to the same. These include full-time employment, single-parent status, having other dependents, being financially independent, enrolling part-time, lacking a high school diploma, and delaying enrollment after high school. 

As one indicator of student success, completion metrics can illustrate the importance of predicted outcomes. Take for example, “graduation rate,” which is commonly understood as the percentage of students who graduate within a defined period. If School A has a graduation rate of 60 percent and School B has a graduation rate of 70 percent, it might appear that School B performs better. However, if the predicted graduation rate for School A was 58 percent and for School B was 80 percent, it becomes clear that School A actually did a better job at graduating its students. 

As the example illustrates, without proper context, graduation rate isn’t a very instructive measurement. Worse, if regulations incentivize high graduation rates across the board, institutions may simply act to raise admission standards or relax academic rigor to avoid penalty. Increasing selectivity may be appropriate for selective institutions, but it is certainly not a national prescription to achieve President Obama’s goal of having the highest proportion of college graduates in the world by 2020. Regulations should not discourage institutions from educating populations of students traditionally underrepresented in American postsecondary education. Instead, institutions ought to be encouraged to accept that challenge and should be recognized for doing so successfully. In addition to evaluating outcomes, this approach may also provide future opportunities to better understand successful practices. “Quality” education means something entirely different for each type of learner. 

While it’s natural that policymakers would seek to reward institutions that “perform” well and to help ensure the taxpayers’ investment in postsecondary education is not squandered on programs adding little or no value, the stakes are indeed high. This is particularly true for our nation’s most vulnerable students, who depend upon the institutions that embrace the real-world challenges facing them and develop programs and support systems to overcome those challenges. Institutions should be evaluated in context and encouraged to operate according to their mission, not to simply satisfy a metric or avoid a penalty. And to be clear, contextual measures are necessary for accurate performance assessments, but should not diminish our goals for unilaterally high performance. Performance must continue to be measured in both actual and contextual terms. Hopefully, Congress and other policymakers will consider this perspective as it deliberates reauthorization of the Higher Education Act in the coming months.

Marcus Szymanoski is the manager of training and communications for DeVry Education Group’s regulatory affairs division. Marcus has worked in postsecondary education for more than ten years, specializing in education policy and student financing. In addition to training, Marcus also performs legislative analysis and outreach and is an active volunteer for NASFAA, MASFAA, and ILASFAA. 

***

What do you think about evaluating performance in both actual and contextual terms? What are unique factors at your institution that may impact some of the metrics? Please share your thoughts below. 

 

Publication Date: 10/20/2014


Theodore M | 10/29/2014 1:11:28 PM

Well said! I have often believed that comparison to mission is one of the major faults in US News rankings. In essence, we are all compared to Harvard's mission (and there is nothing wrong with Harvard's mission) but other schools have a completely different raison d'etre.

Maureen A | 10/29/2014 11:54:57 AM

Very good article Marc! Institutional student profiles vary so far from each other and policymakers need to take this into account prior to developing performance metrics across the board. If it is the institution's mission to provide access to the most vulnerable (e.g. requiring developmental coursework), the success rate at graduating these students will vary from more selective institutions.

Justin B | 10/29/2014 10:12:50 AM

Excellent write-up, Marc. Context is important!

You must be logged in to comment on this page.

Comments Disclaimer: NASFAA welcomes and encourages readers to comment and engage in respectful conversation about the content posted here. We value thoughtful, polite, and concise comments that reflect a variety of views. Comments are not moderated by NASFAA but are reviewed periodically by staff. Users should not expect real-time responses from NASFAA. To learn more, please view NASFAA’s complete Comments Policy.
View Desktop Version