Company

For someone who travels like me – all over the planet working with institutions to develop manageable, sustainable practices for measuring student learning it feels like spring . . . maybe the kind that gets your hay fever going.  It feels like spring because from North America to Asia, home grown assessment models are blooming.  By this I mean that many institutions have invented their own way of number-crunching grades or pass rates and call that measuring program outcomes.  These practices tend to have a common thread running through them; they generate data that may be useful in telling you where you stand, but they do not provide the slightest hint on how to improve.  Grades never did, and grades never will.  The whole point of assessing program learning outcomes is to identify ways to improve and to close gaps between desired performance and actual performance.

Accreditation requirements around the world, by the way, are pretty consistent and clear:

1.  Identify your program outcomes;

2.  Measure your program outcomes;

3. Use the results of measurement to improve your program.

Yes, assignment grades are direct measures, but direct measures of student performance – not program performance.  In individual courses instructors grade holistically and appropriately include both disciplinary knowledge and some combination of trans-disciplinary skills, such as written communication, critical thinking, and/or information literacy.  Moving to the program level, since no two instructors apply grading criteria in the same way, the idea of aggregating grades begins to get my springtime allergies going.  The basic problem with using grades as a measure of program effectiveness is the logic gap between measuring student performance on an assignment in a course and the need to measure how well a collection of courses produces program-specific knowledge, skills, and competencies.   Aggregating assignment grades gives us data, but you don’t have to drill very far down to see that disciplinary knowledge is mixed together with one or more trans-disciplinary skills in any given grade.  Oh, and don’t get me going on assessment beginning to influence instruction in the classroom, which is worthy of its own blog posting!

And one more thing about course grades as a measurement of program outcomes:  this practice may lead to recommendations for improvement, but grade data are never sufficiently granular to know how to improve performance.  Thus, improvement strategies identified under these circumstances are often nothing more than reasonable guesses.  Take critical thinking, for example.  Can a grade tell you what elements of critical thinking need to be improved among its dimensions – identify issues, recognize context, take perspective, evaluate assumptions, evaluate evidence, and evaluate implications?  No.

So how can you make course grades work in measuring program outcomes?  You can’t.  By nature any grade given in a course contains both disciplinary knowledge combined with at least one other trans-disciplinary outcome.  When somebody explains to me what aggregate grades represent, I’ll dismount from my pedantic high horse, take some allergy medication and stop sneezing.