September 28, 2020

Measuring Student Impact Across Time and Effort

This content was previously published by Campus Labs, now part of Anthology. Product and/or solution names may have changed.

Entering a school year that looks very different from what we’re familiar with, there is more of a priority than ever for colleges to create cocurricular experiences that translate to learning and promote the skills employers want. It is not enough to believe we are teaching these skills. Rather it is critical that we are creating experiences that have clear learning goals and teach students how to articulate the learning obtained in these experiences during an interview. To develop their first virtual engagement opportunities, it is important to recognize that it is not enough to be simply providing an abundance of involvement opportunities. Instead, colleges need to improve the quality of learning that results from these experiences before encouraging an increased involvement level.

Structuring Integrating Learning

When most people think about college learning, there is a temptation to think first, and perhaps exclusively of the classroom. But what about the myriad of other learning experiences on campus such as participation in a student organization, intramurals, or service projects?

Student affairs scholar Stanley Carpenter (2012) explains that few think of these experiences as learning. He writes, "It is fashionable now for some to talk about the...co-curriculum and its role in student learning. That is... for those of us in student affairs. For nearly all students, most faculty, and many parents and politicians, this is nonsense. While they understand that students do change and grow emotionally and socially during college, they do not attribute the change to anything other than natural maturation and some vague notion about the college experience." (Collins and Roberts, 2012, p. vii). Many who read this likely find it disheartening, but their own experiences would likely tell them that many still feel this way.

Despite what many may believe, Richard Light (2004) found that “...evidence shows that when students were asked to think of a specific, critical incident or moment that changed them profoundly, four-fifths of them chose a situation or event outside the classroom”. It’s clear that students find their learning outside the classroom to be significant, so how can student affairs practitioners change this perspective?

In pursuit of an integrative learning environment, what should students learn? The recent focus on the skills desired by employers could do more than demonstrate the value of a college education to these businesses and industries that may hire them upon graduation. It can provide a new relevance for student affairs and the hope of a truly integrative learning environment. In 2016, National Association of Colleges and Employers (NACE) used data collected through their job outlook survey as a foundation for creating its Career Competencies. These include critical thinking/problem solving, oral/written communications, teamwork/collaboration, digital technology, leadership, professionalism/work ethic, career management and global/intercultural fluency (Career Readiness Defined, n.d.).

Data from Project CEO demonstrate that students gain both technical and transferable skills from their classes and their cocurricular experiences. Classes tend to be the best source of technical skills – but cocurricular experiences provide a meaningful opportunity to apply the transferable skills they are gaining from their classes. By focusing on transferable skills – students can meaningfully apply skills they are gaining in the classroom.

A common understanding of what we want students to learn is an essential foundation for creating integrative learning. But to fully capitalize on this concept, it is important to determine how this learning can be structured so students can advance what they are learning.

Increasing Learning Across Time

It would be hard to make a case that integrative learning is truly taking place if students don’t learn more as they invest more time and energy. There is a natural structure for this on the academic side. They narrow down their learning to a discipline or major – and the courses increase in complexity and draw upon learning from foundational courses.

But it’s challenging to think about creating a parallel structure in cocurricular experiences because there is no prescribed order to our curriculum. A student’s progression through what we teach them can be a bit more haphazard. Some may only be somewhat involved throughout college – some may start their journey with great zest and zeal and continue this momentum through graduation.

Instead, we can look at structuring student learning based on depth of involvement. In the Cocurricular Career Connections (C3) leadership model, Peck and Preston propose three general stages which mirror the natural process student take to become leaders: “Cocurricular Involvement, cocurricular engagement and cocurricular leadership” (p. 35).

To better understand if investment in an organization correlates to greater articulation in career readiness skills, we looked at our Project CEO data and created segmentation definitions based on the C3 model to see the difference in response between students who identified themselves to be Involved, Engaged, or Leaders. These definitions were created based on questions where students identified their involvement level and whether they held an officer or leadership position in these organizations. Our “involved” students were defined by being somewhat involved in a single or multiple organizations with no officer/leadership position, our “engaged” segment was defined by students who said they were very involved in one or more organization with no officer or leader positions, and our “leaders” segment was defined by being very involved in a single or multiple organizations while being a leader or officer in those organizations.

Using NACE’s 2019 survey that asked employers to identify the importance of career readiness competencies, we wanted to see how our involved, engaged, and leader segments self-reported their ability to demonstrate the three most important skills employers are looking for.

Professionalism and Work Ethic

ethic 1

ethic 2

Teamwork/Collaboration

teamwork 1

teamwork 2

Critical Thinking/Problem Solving

thinking 1

thinking 2

As shown above, in the top three skills that employers are looking for, our leader segments showed a statistically significant difference from the Engaged or Involved segments using a one-sided t-test. Even without much information about how the campuses participating in the benchmark are structuring their engagement opportunities, students report that they are more deeply involved to be more attuned to higher order learning than those who report they are less. This gives us evidence that depth of involvement correlates with both the articulation of skills and confidence in the ability to demonstrate these skills.

The critical element of this model is that as students progress in these broad stages, institutions must write learning outcomes and plan learning activities based on progressing toward higher order learning. For example, learning outcomes for involved students should be written based on what they can “remember and understand.” Engaged students should be able to apply and analyze. And leaders should be able to evaluate and create.

To show how this might work from a practical standpoint, we can look at something most effective cocurricular experiences do – set goals. For an involved student, who may not be contributing much to the group because they are either new or not putting forth much effort – we could only expect that they “remember” and “understand” the group’s goals. For engaged students, since they are doing the group's work, we should expect that they can "apply" the group's goals by finding strategies for accomplishing goals and "analyzing" by looking at how to overcome barriers to accomplishing these goals. Finally, leaders play an important role in facilitating goal development, thus “creating” goals – and should be able to “evaluate” these goals to see how well the group is doing.

The goal of integrative learning is a lofty one – and the ability for experiential educators to contribute to this kind of structure may seem like it is a purely philosophical exercise. However, the framework we have just laid out shows that this work can be approached very practically and intuitively.

Works Cited:

Astin, H. S., & Astin, A. W. (1996). A social change model of leadership development guidebook, Version 3. Los Angeles: University of California, Los Angeles. Career Readiness Defined. (n.d.). Retrieved from http://www.naceweb.org/career-readiness/competencies/career-readiness-defined/

Collins, K. M., & Roberts, D. M. (2012). Learning is not a sprint: Assessing and documenting student leader learning in cocurricular involvement. Washington, DC: NASPA–Student Affairs Administrators in Higher Education.

Dungy, G. (2011, December 23). Campus chasm. Retrieved from https://www.insidehighered.com/views/2011/12/23/essay-lack-understanding-between-academic-and-student-affairs

Dungy, G. & Peck, A. (2019). How campus activities can lead the modern university: Five imperatives, Journal of Campus Activities Scholarship and Practice, National Association for Campus Activities, 1 (1).

Kolb, D. A. (2014). Experiential Learning: Experience as the Source of Learning and Development (2nd Edition) (2nd ed.). Pearson FT Press.

Light, R. J. (2004). Making the Most of College: Students Speak Their Minds (Revised ed.). Harvard University Press.

Mackes, M. (2017). Employer-Preferred Skills and Attributes. In Peck, A. (Ed.), Engagement and employability: integrating career learning through cocurricular experiences in postsecondary education (xi-xxxi). Washington, DC: NASPA-Student Affairs Administrators in Higher Education.

Peck, A. & Preston, M. (August, 2018). “Connecting bridges: Introducing the co-curricular career connections leadership model,” Journal of the National Association of Colleges and Employers.

Seemiller, C. (2014). The student leadership competencies guidebook: Developing intentional leadership learning and development. San Francisco, CA: Jossey-Bass.

Headshot of Rohan Thakkar

Rohan Thakkar

Enterprise Consultant, Education and Adoption
Anthology

Rohan Thakkar currently serves an Enterprise Consultant, in which he is responsible for producing a variety of content, resources, and expertise focused on institutional assessment. In this role, he also oversees the Assessment Credential Certificate, leads the survey consultation team and works closely with an assortment of campuses in their use of the Anthology solutions. Prior to Anthology, Thakkar served in an array of higher education programs that have yielded expertise in divisional strategic planning, learning outcomes assessment, student affairs practice, professional development in assessment competency and data analysis. He received his undergraduate degree from Rutgers University – New Brunswick and has his master’s degree in college student personnel administration from Seton Hall University.

Adam Peck, Ph.D.

Assistant Vice President, Student Affairs
Illinois State University