Student Evaluations of Teaching are Not Valid

The author misses a very significant fourth reason for using SET scores: consumer happiness. A happy consumer continues to consume the company's product. By the standards of business, it is good to maximize immediate satisfaction. If the academy continues mistaking education for business, then SET scores will become ever more important. If we return to the discarded educational model for higher education (not sure why we don't now call it "higher business"), learning outcomes become primary. Under the business model, they remain secondary. Nice to learn but not necessary. Until we, through our legislators, once again support higher education through tax revenue, schools will have to favor tuition revenue over learning.

I left academia a long time ago, but I can't help recalling a year when I taught the same subject in two different sections at a large state university, using the same materials, moving at the same pace, and making every effort to engage the students. One section rated me highly, with glowing comments, and the other section rated me poorly, with sarcastic comments.

So was I a good teacher or a bad teacher? The second class had a high percentage of what my colleagues called "the baseball cap crowd," young fraternity and athletic types, while the first class had more women, international students, and older students.

This kind of thing happened again and again in my 11 years in higher education.

As I look back over my own student days, I know that I clearly had some bad teachers, by which I mean the ones who never prepared and just b.s.'d their way through class or the ones who spent the hour reading aloud from the handouts that they had just given the students. I also had some superb teachers who made everyone interested in subjects that are commonly assumed to be boring.

But there's a huge cohort in between the awful and the superb, and it is hard to know how to evaluate them fairly. Perhaps it is best to mentor the underachievers or the uneven performers within a department by partnering each new instructor with an experienced instructor and holding department-wide meetings in which faculty members present their most successful teaching techniques.

As both a former student and former teacher (now retired), I believe student evaluations can be important both for evaluating teachers and helping them improve, but ONLY if they are open-ended (essay), not multiple choice (rate from 1 to 7). Open-ended evaluations allow one to see WHY students give the ratings they do. Unfortunately, most utilizers of student evaluations (i.e. administrators and sometimes faculty peers) don't really care enough to spend the time assessing USEFUL evaluations, but prefer the pointless & stupid multiple choice type, which allows obtaining a numerical score with little effort (or meaning). Of course this is why every company in the world is administering such stupid evaluations after every transaction: they don't really care about quality or WHY you are happy or not, they just want an easy number to get (which, among other things, facilitates getting ridding of "trouble makers" if they're looking for some excuse--which they are!).

The AAUP Committee on Teaching and Research conducted a nation-wide survey of faculty members' experiences with and views on student evaluations of teaching in 2015. The information that we gathered and the suggestions that we made, which were for the faculty cohort, usually the department, to define profession-appropriate standards for coursework and to support one another in achieving them, to provide mentoring and development opportunities for all faculty members, to share methods and solve problems together, and to conduct faculty-based evaluations of teaching were published in: "How do We Evaluate Teaching? Findings from a Survey of Faculty Members." Academe 102.3 (May-June 2016): 34-39 https://www.aaup.org/article/how-do-we-evaluate-teaching#.Ww9AMCBlDIU (Craig Vasey and Linda Carroll; based on the work of the Committee on Teaching, Research, and Publication of the American Association of University Professors); cited by Colleen Flaherty, “Flawed Evaluation,” in Inside Higher Education, June 10, 2015; Michelle Falkoff, “Why We Must Stop Relying on Student Ratings of Teaching,” in The Chronicle of Higher Education, April 25, 2018 (distributed by the Modern Language Association to its members in an email message of May 16, 2018); Colleen Flaherty, “Teaching Eval Shakeup,” in Inside Higher Education, May 22, 2018.

Is your university rethinking evaluation of teaching? Please complete this short survey to help us understand your perspective.

https://baseline.campuslabs.com/wsu/evaluationofteaching

Bravo to the authors for pointing out the bankruptcy of taking course evaluations too seriously. I have argued at my school that after we come to some consensus on what specific content we are trying to teach, we should prepare written or oral comprehensive exams or other similar student assessments to see if we succeeded. The consensus should be at the department or school level to abstract away from individual teacher's characteristics or preferred topics. The question then should be whether they have internalized the content, rather than whether they were happy trying.

I enjoyed reading the discussions about SET. After 52 years of haematology-oncology practice in an university and teaching I have retired, but, continue teaching. There is no easy way to become an out standing teacher in the eyes of students. Evaluation is very subjective. Less demanding spoon feeding teachers get good scores, particularly near the time of examination. Problem based tutorials rather than lectures are more effective.

Readers may be interested in this article: https://ocufa.on.ca/blog-posts/significant-arbitration-decision-on-use-o... . It provides links to the Freishtat and Stark reports, as well as the arbitration at Ryerson University in which they were evidence.

Performance in subsequent courses seems the closest thing to an objective evaluation of teaching effectiveness. This seems most obvious in multi-semester courses, but in those courses seen as foundational, one should presume that effects should affect multiple parts of the student's curriculum. Teachers whose students consistently underperform or overperform on a GPA basis compared to teachers of similar student cohorts (no comparing ESL or developmental English teachers to accelerated English teachers) would seem to be worth examining for faulty or exemplary methods.

Faculty who survive this poor and ridiculous practice of emphasizing student evaluations as a measure of teaching effectiveness must game the system. There are a variety of ways but for sure it involves reducing the workload of the courses and inflating grades. In doing so, they must take a hit to their sense of academic integrity (assuming they have some). Students are also big losers in this debacle. Though they have a degree, they lack the commensurate skill which undermines gainful employment and many life outcomes. Parents and others contributing to the college or university system are also stiffed.

I'd like to look at them and perhaps forward them to administrators where I teach.

Thanks John for your article, and all of you for contributing to this page. If it is not too late, I would like to add a few things, none of which I'm sure are news to you, but which are nevertheless nice to see in writing. I am a Mathematics professor. The field does not change much. 1+1 was 2 eons ago, as it is today, as you can imagine. The way you teach Math is simple, you show all the work, and it has to be transparent. Problem: students just want the answer. Negative points for you. Students have to practice, and it's hard, nothing is served on a platter. Problem: in my other classes, I constantly get A's, why would I have to put in some effort in your class and get by with a C at best, the thinking goes. You can easily see that our evaluations are not good. Actually, they are the worst on a campus, but not necessarily so. You can turn into a circus animal, entertain your students, give take home finals, not grade homeworks. Actually, scratch that. Don't give homeworks. Water down the material. Essentially then, teach by your evals. Use them as a yardstick. Most typically, your evals will be very high, as I have observed over the years with other faculty whose evals were stratospheric. Those students move on, we inherit them, they know nothing, they hate you for it, and they bury you. What do you do really? You have to survive, so you just give in to what I call high end prostitution, whereby you completely ditch your standards, and you behave like an entertainer, because the client is always right, and that's what the client desires. Today, that's what I'm starting to do. I would not hire a single one of my students, for the simple reason that I adapted my teaching to their liking, and I do not form good students anymore, specifically because I paid attention to their litanies of demands. They hit the job market, and they are wholly unprepared, employers are absolutely appalled. Who's at fault really? The system is perverse, it forces us to teach by the evaluations. One last point. Going back over my evals, I hardly see constructive criticisms. It really does not help in improving your teaching per se, at least, not in Mathematics. I only use them to please the clientele, and every time I do that, I lose in quality. This is all very sad. My evals have gone up over the years, but not the quality of education I proffer. Actually, it's inversely proportional.

December 2022, we haven’t made any progress on this yet.
It is unfortunate that professors put on all of this teaching effort and small private universities ignore the fact that young, female and non-white people are rated much poorly than other professors. This, of course, has a huge impact on mental health. It really does encourage mediocre teaching (like a white male colleague of mine who just plays movies to students because that gives him better scores), and grade inflation. I decided to not ask my students to complete evaluations anymore and I also decided to put my mental health first by never reading the evaluations. In my institution you can see who rated you and it’s always the bad students in your class the first ones to leave mean evaluations. The whole semester effort is summarized in negative commentary. In psychology, you do better with encouragement and positive and helpful feedback, not with detrimental comments and unfounded criticisms. It is sad that in my institution, only 30% of the professors are female and the director for assessment does everything in her hand to make the institution look like our students are all prodigies (thanks to grade inflation).