Beyond Course Evaluations - Constructive Feedback without Wishful Thinking

I aspire to be a professor my students remember - years from now - as someone who inspired them. Yet when we're judged, it's with course evaluations presenting us like short-term service providers with course objective "products" to sell. I didn't choose a career teaching to "assess" my students.

Some colleagues suggest I’m not "SMART." Specific measurable learning goals strike me as just schooling, not education. I model myself after teachers who profoundly mattered to me. I teach to help my students grow and find their own paths through life.

“How do you measure that?”

Honestly, that’s a good question. Could I be deluding myself with my own wishful thinking? I get high ratings, but maybe I’m self-handicapping to protect myself just in case someday I don’t? Without data from course evaluations and measures of specific learning objectives, how can we confident in ourselves as effective teachers? How can we objectively measure our progress in ways we see as really mattering?

I grow my teaching with seeds planted in me by my teachers, and each semester I try to improve my teaching with honest reflection with evidence I gather over the semester. With so much focus on bubbles students choose on evals, it's easy to miss just how much evidence naturally appears while we teach.

Honestly look at what students say, but more importantly, observe their behaviors like body language and micro-expressions (subtly quick emotional reactions on the face). When do they look like we’re making them do chores and when do they show anticipation? “When” not “if” because we constantly get lots of affirming and dismissing feedback. Some topics we teach are just more exciting than others, and it’s okay if we need to teach something less engaging. Some students are just sullen, enthusiastic, or maybe just tired. But whatever their baseline, reactions shift. Why?

I notice with breakout student discussions I can increase enthusiasm, but sometimes anxiety too. Over time I came to realize, if discussion require ‘expertise’ like abstractly applying what we’re learning, students look for evidence they’re “right.” But when I make breakouts about personal experiences with only a gentle nudge to connect, I can use their relaxed enthusiasm. I can build bridges when we come back together into examples they find meaningful as I connect them back to our topic.

Our trial-and-error literally builds day-after-day, and semester-after-semester, without us even realizing it! I taught in a “fishbowl” classroom with a glass wall to the “great room” where people regularly walked passed. A colleague mentioned how she always sees me sitting upon my desk and, to her, it looks like I’m chit chatting with my students over coffee rather than lecturing.

I realize since I started teaching, I keep getting more personable and I share lots of short personal stories, often self-deprecating or funny - all of which illustrate ideas I’m teaching. Reflecting on myself, I read research discovering how much evidence there is for improving students’ memory, attention, and depth-of-processing with story-telling. I did studies, and literally published a book chapter about it (Grobman, 2015).

Intuition is not the opposite of analytical thinking even though analytical thinking is more common in high IQ people (like professors) and intuition is negatively correlated with IQ (e.g., Alaybekl et al., 2021; Hodgkinson & Sadler-Smith, 2003). Experts like nurses, firefighters, and car mechanics give more accurate responses relying upon their implicit learning than when required to make their judgements explicit (e.g., Chassy & Gobet, 2011; Gore & Sadler-Smith. 2011). And intuition predicts performance better than analytical reasoning with less well-defined problems requiring creativity under time-pressure. To me, that’s teaching. As long as we’re honestly reflecting and honing our expertise, intuition is extraordinarily powerful - Intuition and good judgement are analysis frozen into habit (Simon, 1987).

Maybe we shouldn’t worry so much about measures? Nevertheless, practically, we need to show evidence of our teaching to people in authority. And if we have scientific evidence supporting our intuitions, our ideas become more compelling when we share. As a scientist, reliably measuring is how we know. Our research methods and statistical analyses are meant to help us avoid biases, like wishful thinking, and bring us towards an objective understanding of reality. It’s worthwhile designing quantitative measures of teaching. But can we focus on measuring what matters?

One difficulty with typical course evaluations is we don't know who provided each response. Was disheartening feedback a disgruntled student upset I expected attendance or a conscientious student expressing frustration? When we have some positive and some negative feedback, were there any patterns suggesting we helped students of some backgrounds more than others (e.g., gender, ethnicity, political identity, major, year in college, GPA)? If only we could match each course evaluation response with each student in our grade-book. Unfortunately if you ask the usual way, students naturally feel uncomfortable saying something negative to the person who assigns their grade.

I've developed a measure (figure 1) that works even when students are not anonymous. I use it to consider individual differences within class. More powerfully, I compare responses across semesters in the same class seeing how much pedagogical changes I make impact learning.

Please feel free to use it for your personal improvement! Compare your teaching across semesters. Just please don't use it to compare faculty or across courses because norms invariably differ.

Students are not evaluating our class, but providing feedback about their behavior as a consequence of our class. Hopefully their responses are honest self-appraisals rather than thoughts about us. In contrast with my course evals where the most common response is the highest value (i.e., ceiling effect), to most students, it feels unrealistic to answer “multiple times a week,” so they readily choose lower numbers. Some choose “multiple times a week” and this is consistent with my conversations with students most enthusiastic about their learning. All five items load on a single factor, suggesting they measure the same thing (i.e., depth of understanding class material). However, no two items are correlated beyond about .7, suggesting they each tap something subtly different. Consistent with what we might hypothesize, my statistical analysis shows higher ratings come from students higher in need-for-cognition and with higher grades.

When scoring, I code responses to an approximate number of occurrences per semester, as follows: 1 becomes 0, 2 becomes 1, 3 becomes 3, 4 becomes 6, 5 becomes 9, 6 becomes 15, and 7 becomes 30.

Please be cautious about comparing your averages with other professors or even across your own courses. Sometimes the content of particular classes is more amenable to fostering particular items. It's okay. We're not using the tool to judge ourselves with comparisons, it's about our own growth semester after semester. Even so, if you would like a comparison, my typical Introductory Psychology averages are from "majority of weeks" to "nearly every week" (~12) and my typical Social Psychology averages are "nearly every week" (~14). Please remember teaching to deeply inspire students has been my guiding light for years and if you’re new or you have focused on something else, like learning objectives, your score might start lower. It’s about charting your own journey toward your guiding light.

Honing our goals with learning objectives and reflecting on our students reactions on course evaluations helps us grow into competent teachers. But if you aspire to something greater and intangible, you can. When we’re honest observers of our own teaching and hone our skills like the experts we are, we can become an inspiring force in our students’ lives. And there are ways we can gather data to showcase to others our accomplishments.

Alaybek, B., Wang, Y., Dalal, R. S., Dubrow, S., & Boemerman, L. S. G. (2021). Meta-analytic relations between thinking styles and intelligence. Personality and Individual Differences, 168(1), 110-322.

Beilock, S. (2010). Choke: What the secrets of the brain reveal about getting it right when you have to. Simon and Schuster.
Chassy, P., & Gobet, F. (2011). A hypothesis about the biological basis of expert intuition. Review of General Psychology, 15(3), 198–212.

Gore, J., & Sadler-Smith, E. (2011). Unpacking intuition: A process and outcome framework. Review of General Psychology, 15(4), 304–316

Grobman, K. H. (2015). Antsy students impatient to leave class and faculty captive in NPR driveway moments? Enhancing science classes with personal stories. In K. Brakke & J. Houska (Eds.), Telling stories: The art & science of storytelling as an instructional strategy (pp. 98-115). American Psychological Association Society for the Teaching of Psychology.

Hodgkinson, G. P., & Sadler-Smith, E. (2003). Complex or unitary? A critique and empirical re-assessment of the Allinson-Hayes Cognitive Style Index. Journal of Occupational and Organizational Psychology, 76(2), 243-268.