Evaluations

By a collective decision of the faculty of Grinnell College, we now administer the same student-evaluation form in all of our courses except the first-year tutorial. The form presents six assertions, oddly referred to as “questions”:

1. The course sessions were conducted in a manner that helped me to understand the subject matter of the course.

2. The instructor helped me to understand the subject matter of the course.

3. Work completed with and/or discussions with other students in this course helped me to understand the subject matter of the course.

4. The oral and written work, tests, and/or other assignments helped me to understand the subject matter of the course.

5. Required readings or other course materials helped me to understand the subject matter of the course.

6. I learned a lot in this course.

The student is invited to disagree strongly, disagree moderately, disagree slightly, agree slightly, agree moderately, or agree strongly with each of these assertions, or to decline to express an opinion (“Not Applicable/Don't Know”). The Office of Institutional Research tallies the responses and returns the tallies to the teacher of the course.

In the first-year tutorial, we use a similar but slightly more elaborate form, inviting students to agree or disagree with a total of eighteen assertions.

The faculty approved the use of the results of these evaluations for only one purpose: When a faculty member is being reviewed (for retention, promotion, or tenure), the Personnel Committee receives the results of his or her evaluations. However, factoids derived from the evaluations are equally useful -- indeed, exactly equally useful -- in many other contexts. At the suggestion of the President of the College, therefore, I have made the results of evaluations of my courses available for whatever purposes readers may conceive. I invite comments and suggestions from interested readers.

The OIR also computes various additional statistics from the tallies by awarding points for each response according to the following table:

Using these scores, the OIR computes a 95% confidence interval for the supposed mean score for each question, and sometimes includes these in its reports to teachers. Because computing such intervals exemplifies the elementary statistical fallacy that is explained in my article “Scales,” I have not bothered to reproduce them here.

• CSC 301.01, “Analysis of algorithms,” fall 2012
• LIN 114.01, “Introduction to general linguistics,” fall 2012
• CSC 323.01, “Software design,” spring 2012
• CSC 341.01, “Automata, formal languages, and computational complexity,” spring 2011
• CSC 205.01, “Computational linguistics,” fall 2010
• CSC 301.01, “Analysis of algorithms,” fall 2010
• CSC 362.01, “Compilers,” spring 2010
• LIN 114.01, “Introduction to general linguistics,” fall 2009
• TUT 100.29, “Free software, free culture” (tutorial), fall 2009
• CSC 302.01, “Programming language concepts,” spring 2009
• LIN 114.01, “Introduction to general linguistics,” fall 2008
• CSC 205.01, “Computational linguistics,” fall 2008
• CSC 152.01, “Fundamentals of computer science II,” spring 2008
• CSC 302.01, “Programming language concepts,” spring 2008
• CSC 341.01, “Automata, formal languages, and computational complexity,” spring 2007
• CSC 205.01, “Computational linguistics,” fall 2006
• TUT 100.29, “Free software, free culture” (tutorial), fall 2006
• CSC 341.01, “Automata, formal languages, and computational complexity,” spring 2006
• CSC 151.01, “Fundamentals of computer science I,” spring 2005
• CSC 341.01, “Automata, formal languages, and computational complexity,” spring 2005
• CSC 205.01, “Computational linguistics,” fall 2004
• CSC 151.01, “Fundamentals of computer science I,” spring 2004
• CSC 341.01, “Automata, formal languages, and computational complexity,” spring 2004
• CSC 261.01, “Artificial intelligence,” fall 2003
• CSC 302.01, “Programming language concepts,” spring 2003
• CSC 205.01, “Computational linguistics,” fall 2002
• CSC 302.01, “Programming language concepts,” spring 2002
• CSC 364.01, “Computer networks,” spring 2002
• CSC 151.01, “Fundamentals of computer science I,” fall 2001
• CSC 341.01, “Automata, formal languages, and computational complexity,” spring 2001
• CSC 301.01, “Algorithms,” fall 2000
• CSC 151.01, “Fundamentals of computer science I,” spring 2000
• CSC 223.01, “Software design,” fall 1999
• CSC 301.01, “Algorithms,” fall 1999

For comparison, here are the college-wide aggregate results. (However, the Office of Institutional Research compiled at least some of these tallies by a method that yields incorrect results. The data in these tables should therefore be taken as approximate.)

• Fall 2012
• Spring 2012
• Spring 2011
• Fall 2010
• Spring 2010
• Fall 2009
• Fall 2009 (tutorials)
• Spring 2009
• Fall 2008
• Spring 2008
• Fall 2007
• Spring 2007
• Fall 2006
• Fall 2006 (tutorials)
• Spring 2006
• Fall 2005
• Spring 2005
• Fall 2004
• Spring 2004
• Fall 2003
• Spring 2003
• Fall 2002
• Spring 2002
• Fall 2001
• Spring 2001
• Fall 2000
• Spring 2000
• Spring 1999

In interpreting evaluations from previous years, readers may wish to take into account the secular inflation of ratings, particularly on questions 2 and 6. Note, for instance, the rise in the percentage of students giving the response “strongly agree” to each question:

Readers may also wish to note that both the bandwidth of the quantitative part of the evaluation form (the upper bound on the amount of information that can be transmitted through that channel) and the information content of the actual responses are surprisingly small.

Each student's response to each question is assigned to one of seven groups (a group for each of the six scalar responses and a combined “not applicable,” “don't know,” no-response group). The bandwidth is therefore log₂ 7 (approximately 2.8) bits per response, for a total of 6 log₂ 7 (approximately 16.8) bits for the entire form.

However, our students' actual responses do not use this bandwidth with perfect efficiency, since not all of the responses to a question are equally likely. For each of the six questions, the distribution of responses is heavily and predictably skewed towards the “Strongly Agree” and “Moderately Agree” groups. Hence the mean information content of those responses is less than the theoretical maximum. In addition, the mean information content has diminished over time, as secular inflation has tended to make the distributions even more skewed.

The following table shows the mean information content (in bits) of responses, broken down by question and by semester. The “Total” column shows the mean total information content of all six questions in the specified semester.

For comparison, the information content of written English prose is generally estimated to be at least 5.5 bits per word. Thus one set of responses to the quantitative part of the evaluation form could in principle carry about the same amount of information as three words of prose, but in practice conveys only about two words' worth, on the average.

Professor Rebelsky's essay “About end-of-course evaluations” explains the purpose and recent history of student evaluations at Grinnell in greater detail. (It's also more interesting reading than this page.)

This document is available on the World Wide Web as

http://www.cs.grinnell.edu/~stone/evaluations/

created January 18, 2001
last revised January 30, 2013

John David Stone (stone@cs.grinnell.edu)