The Civic Incomes Project

Begun in 2024, Civic Incomes is also a new way to look at student learning assessment, shifting from an outcomes-only approach to a more holistic and longitudinal one. 

The Need

Civic engagement programming assumes all students are well-prepared to conduct acts of informed civic leadership from their first day.  However, while some students will enter with an existing skills base, not all students will.  Students come in with varying levels of civic knowledge.  American Democracy Project participants have long striven to develop ways to better tailor civic engagement to students’ abilities.  Assessing the civic capacity of students when they enter a college career is a method that bears great promise in aligning civic engagement activities with student skills and needs.

Assessment of student learning outcomes is an established practice, but few if any institutions systematically assess the level of knowledge and skills with which students begin their collegiate experience.  Assessing students’ baseline of civic knowledge would allow universities not only to strategically align our civic work with building those skills, but they would allow us to discern the longitudinal growth of those skills over the course of a students’ matriculation.  In effect, an incomes assessment would allow us to quantify the value-add of a college degree from the difference in scores from income to outcome assessment. 

The Project

Over the course of 2024, a team of six faculty and staff members at Radford University developed an instrument to assess student civic preparedness upon their arrival.  The team convened beginning in February of 2024.  I provided several externally produced assessment instruments to the team for consideration.  The advantage of adapting existing instruments is primarily to allow for external validity of the data we collect.  After consideration, the team decided to adapt the Inter University Consortium for Political and Social Research’s (ICPSR’s) Political Engagement Project (PEP) instrument initially developed in 2005.  The team met and collaborated on reducing the more than one hundred response prompts in the PEP to a more manageable 21-question survey that could be administered in a single one-hour period. (Attached)

During those deliberations, the team considered several deployment options.  While some of the team’s preference was for administration during New Student Orientation, Director Lucas indicated that the schedule was already too overloaded to effectively deliver the instrument to students.  Lucas offered a class period during the UNIV 100 course, which provides us with the opportunity to reach more than eighty percent of all incoming students.  The team decided that the UNIV 100 deployment would be the best available option, and the instrument will be administered during Fall 2024’s UNIV 100 class schedule. 

The team successfully secured approval from Radford University’s Institutional Review Board (IRB).  Since this data will be made public and could contribute to research, it was imperative to seek and secure IRB approval.  On May 2, 2024, the IRB approved this project with the reference number 2024-075 (Attached). 

The team thus completed all necessary steps to deploy the instrument.  The team organized the questions on the instrument into a four-criteria system to measure different elements of civic engagement:

1)      Civic Knowledge (Basic understanding of political rules, norms, and current information)

1)      Do you read about political events in news sources?

2)      Discussing current events with friends

2)      Civic Dispositions (Attitudes and orientations towards politics)

1)      Ability to take leadership

2)      Fairness

3)      Conflict resolution

3)      Civic Efficacy (Belief that one can make change through legitimate political participation)

1)      Respect for political norms

2)      Patriotism and loyalty to the nation

4)      Civic Engagement (Acts of citizen leadership)

1)      Volunteering for campaigns

2)      Attending local government meetings

The team used a five-point scale of incoming civic capacity based on responses to the assessment, with 1 being the lowest level of development and five being the highest. 

Expected outcomes

·         Creation of an instrument to be deployed to all incoming students measuring their level of civic knowledge and skills

·         Assessment of current participation in civic engagement as a high-impact practice in the classroom

Pilot Results

In the Fall 2024 semester, the team deployed the civic incomes assessment in Radford University’s UNIV 100 orientation class. 684 students completed the assessment, for a total of 86% of all new students at the university.  Student identifying information was collected because we plan to administer this instrument once again, as the student nears graduation.  That will provide a longitudinal measure of student civic capacity growth, which the student can use as a skill artifact when seeking jobs.  For the university, the delta from income to outcome is a quantification of their civic growth over the course of their university career.  In essence, we have provided a measure of the value-add that higher education provides to students. 

Student scores averaged from a low of 2.984 overall on civic knowledge to a high of 3.552 on civic dispositions.  This indicates that while students feel confident in their ability to engage with their communities, they may not have all the foundational preparation needed to fully activate that interest.  As a result, the university’s civic engagement team is shifting some of its work towards more basic educational activities and less in public activations. 

Partnering

The information Radford has collected on our students is useful and can be for any higher education institution.  Therefore, I am seeking to expand our partnership base to other two-year and four-year higher education providers to gauge interest in deploying the Civic Incomes instrument at their campus. 

With a developed instrument and IRB approval, the instrument is plug-and-play.  A university that wishes to participate in the project needs to commit to the following:

·         Securing the opportunity to deploy the Civic Incomes instrument in some form of new student intake, whether that be a pre-semester orientation or during a class for new students.  The survey should be deployed to new students, once a year and in the fall semester at the latest.

·         Securing a second deployment of the instrument for students who are nearing graduation, such as a capstone course.

·         Agreeing to allow principal investigator Chapman Rackaway access to their institution’s data.

·         Some university IRBs will require a separate internal review process, in which case the campus lead will need to successfully seek their IRB’s approval.

·         Secure approval of any other internal entities as appropriate.

Institutions that agree to participate in the Civic Incomes study will receive

·         A copy of their institution’s data each year.

·         Access to the nationwide network of participants in the Civic Incomes project.

·         The opportunity to serve as a co-author on presented and published work using the data.

 Questions

Should you have any questions, please direct them to the principal investigator, Chapman Rackaway, at crackaway@radford.edu.