Assessing Civic learning incomes at Radford university

In assessment, measuring student learning in terms of outcomes is a common practice.  However, it is incomplete.  Student learning outcomes as an assessment measure assumes that either all students come into their college experience with a similar level of skills and knowledge, or that the differences between those skills are irrelevant for purposes of a university education.  I believe that to fully understand the college experience and the benefits of it, we must also understand the level of knowledge and skill those students entered their matriculation.

Through my work in civic engagement, I have seen this manifest regularly.  Universities, meaning well, thrust students who are unfamiliar with the basic differences between the three branches of government into high-level activities such as critiquing city commission meetings or leading voter registration drives.  The result has often been frustration for students, and bad outcomes of the civic engagement activities.  Over time, I came to see that the assumption that all students began their college experience ready to lead was overoptimistic.  

The college experience does require us to get students to the point where they can be those effective citizens, so it becomes important to remediate with civic learning.  The way many institutions glean from their students about their level of civic knowledge is anecdotally, and while it is better than nothing it is not enough.  I have long thought that within civic engagement, and indeed beyond, we need better measures of student incomes so we can tailor skill-building and interventions accordingly.  

If we in higher education could develop a robust system that measures student knowledge at intake and then subsequently iterated that assessment later in their college experience, we could also develop a longitudinal measure that quantifies the value-add of a university education.  A student who would take the same assessment on intake, again at the conclusion of their general education experience, and one more at graduation would provide three data points, the delta between which would quantify the growth that student experienced during their Bachelor's Degree.  

In 2024, I had the opportunity to lead a campus team at Radford University on just such a project.  With the support of Provost Bethany Usher, I assembled a team of civic engagement, curricular, and assessment experts to develop a civic incomes instrument as part of my AASCU Emerging Leaders Program portfolio. 

During the Spring of 2024, we selected the ICPSR Political Engagement Project to adapt to an instrument that would measure students' knowledge, skills, orientations, and experience in civic leadership and secured IRB approval to conduct the project.  Students enrolled in the university's orientation class, UNIV 100, would take the survey during the sixth week of the Fall 2024 semester.  

The Instrument

Results