Measuring Undergraduate Innovation

Can we measure the capacity of undergraduate students to be innovative? In a recent study, Selznick and Mayhew have come up with a theoretical framework, a survey tool, and some initial analysis. Here’s the title, abstract and citation.

The authors (citing others in the field) define innovation as “the generation and execution of contextually beneficial new ideas”. The educator’s challenge is how to construct a learning environment to boost this capacity in students, and concomitantly, how to assess if progress has been made. In particular, the authors wanted to provide a construct that goes beyond the typical references to “profit-motivated innovation” such as entrepreneurship and market-oriented outcomes, and find some way to measure the development of innovative capacity.

After the requisite literature review, Selznick and Mayhew, present the framework of their study. Development of an individual is conceived in three dimensions, not necessarily independent of each other. There is (1) an intrapersonal component that encompasses personal motivation, being proactive, and something called ‘self-concept’, (2) a social component encompassing networking ability, persuasive communication, teamwork; and a (3) cognitive component encompassing creativity, risk-taking, and demonstrating innovative intentions. A summary of the framework is provided below with a sample item in each sub-category. Full details of all the survey items can be found in the article.

The survey data came from ~1400 graduating seniors from six institutions (public flagship, private research-intensive, private liberal arts). However, the pre-test data was much more limited – a group of undergraduate RAs. This was used to refine the survey questions and the survey instrument as a whole, however they are not a representative sample as the authors acknowledge. The authors mention the importance of additional longitudinal studies to further test the effectiveness of the survey instrument, and of course the subsequent results might be of interest!

Here are some tentative correlations from the preliminary study. When compared to two traits in the Ten Item Personal Inventory (TIPI), extroversion and openness to new experiences, there was some correlation. It’s unclear if comparison was checked for the other eight traits. The main meat of the study utilizes “latent trait theory and polytomous item response models”. (There are further references in the paper for the interested reader; I admit I don’t completely understand them.) The main finding is that innovation capacities lined up most strongly with “two intrapersonal constructs (i.e., proactivity, self-concept) and two cognitive constructs (i.e., intentions, creative cognitions) and less related to constructs comprising the social dimension.” The authors speculate that this has something to do with “social exchanges [being] more context dependent”.

The authors suggest several implications of their work. First, it provides a potential assessment tool that is relatively quick, easy to use, (semi-)validated, and grounded in theory. It complements “the many efforts student development scholars take in bridging theory to practice”. One interesting point they bring up is that “this study shows partial evidence that creative thinking is conceptually and empirically different that critical thinking, at least to some degree… developing critical thinks may not necessarily produce innovators.” I think this is an important distinction. I also find their study both timely and interesting, and I’ll have to read a bit more about the theoretical framework and the methodology since I’m not well versed in either. It also made me think about what sorts of exercises or class activities might promote at least the cognitive constructs; I’m not sure what to do about the intrapersonal constructs yet.