The Challenges

Societal goals for education today include supporting diverse learners, their developing the “practices” of scholarship in different disciplines, cultivating their “creativity, innovation, and passion." But these are difficult goals for an educational system that was designed for information delivery and standardization.

 

Students are not standardized

Cognitive science has a strong history of developing standardized instruments for measuring quantities of interest—intelligence, aptitude, and achievement—and what starts in research often ends up in practice. Now scholars now recognize limitations in the instruments’ validity: Students are not standardized, and instruments developed and tested with aggregate favor the statistical modes. But there is nothing yet to take their place. The result is bias against those who are outside the mode.

 

Key phenomena of learning are idiosyncratic

To study students’ creativity, innovation, and passion, or how they engage in disciplinary practices of learning, researchers need to examine what happens in situ. That means qualitative data capturing what students say, do, write, and make—innovation is not something research can standardize. It is, or should be, a wonderful thing, for students to come up with novel ideas that the curriculum did not anticipate!

 

Large-scale studies have required simple, quantitative measures

Research in the cognitive and learning sciences has long had a choice: Study the rich complexity of particular moments, focusing on small numbers of students, or aggregate simple measures across large numbers of students to produce generalizable findings with statistical power.