The Office of the Provost and the University Senate are currently working together to critique and revise our entire teaching evaluation system. Recent research suggests that student ratings may not accurately reflect the quality of teaching due to biases and other factors [3, 4, 5, 6]. The University of Oregon’s own assessment of student course evaluation ratings have corroborated these findings .
The Association of American Universities (AAU) and other universities around the globe from University of Colorado, Boulder to University College London, England have argued that it is time for universities’ practices regarding teaching excellence and evaluation to align with their policies [1, 7, 8]. As such, the University of Oregon seeks to develop a holistic new teaching evaluation system that does more than simply replace problematic evaluation instruments so that we can help the UO community more effectively define, develop, evaluate, and reward teaching excellence. More specifically, we want to increase equity and transparency in the use and efficacy of teaching evaluations for merit, contract renewal, promotion and tenure, while simultaneously providing instructors with tools for continual course improvement.
The Continuous Improvement and Evaluation of Teaching System
There are three pillars to the University’s commitment to continuous improvement in teaching that are part of the evaluation of teaching system. These include student feedback, self reflection on the part of the instructor, and peer review. We are working to evaluate and improve the tools we use in each of these categories in order to develop teaching excellence and ensure that instructors receive useful, actionable, and constructive feedback.
Student feedback is an important part of the continuous improvement and evaluation system because students are at the center of the teaching endeavor and have a unique perspective to share. Students also have the potential to provide instructors with helpful information to clarify what has been especially beneficial to their learning experience, and what has been a hindrance. However, our current Student Evaluations of Teaching (SETs) or Course Evaluations need improvement, which has led to several initiatives for change.
As a result our first endeavor, the Senate recently approved a new Midterm Student Experience Survey for continuous teaching improvement, but not formal evaluation. The feedback will not be viewable by unit heads or committees, only the instructor. Instructors can use this tool to check student experience against instructor expectations, acquire a sense of class climate, and consider adjustments to class plans if appropriate. This survey will be available campus-wide following a 2018-19 incremental ramp-up using the CollegeNET software, starting with test units. We aim for campus wide deployment by Fall 2019.
Our second project involves pilot testing for an End-of-Term Student Experience Survey to replace the current student course evaluations. The 2018-19 Community for Accelerating the Impact of Teaching (CAIT) on Teaching Excellence and Evaluation and the Continuous Improvement and Evaluation of Teaching Senate Committee will move this work forward. The focus will be on utilizing best practices from the scholarship of teaching and learning, especially to avoid simple quantitative ratings, which have the greatest risk of bias. Unlike the midterm student feedback survey, this tool is being designed for both continuous teaching improvement and formal teaching evaluation.
A crucial part of teaching evaluations is the instructors’ reflection on their own teaching. The UO believes self-reflection should be a regular part of instructors’ course planning and development.
To facilitate on-going self-reflection, the senate approved a new 10-Minute Instructor Reflection. The tool makes it easy for instructors to archive in CollegeNET at the conclusion of a course what went well and what might be improved in the future. As instructors accrue reflections, they can use them to assess the success of various teaching methods, helping guide their decisions regarding which practices to keep and which to change. It also provides a new mechanism for the instructor’s own voice to inform evaluators’ interpretation of student feedback.
In order to improve upon peer review’s efficacy as a tool for continuous teaching improvement and evaluation, the 2018-19 CAIT on Teaching Excellence Evaluation and Senate Committee is working on a Peer Review Framework. This framework will help departments develop practices that are aligned with the criteria for teaching evaluation, and ensure peer review is a useful tool for both improvement and evaluation.
If you would like to learn more about our current teaching evaluation system, please visit the Teaching Engagement Program website.
Evaluation of Teaching Criteria
Revising our evaluation tools is only one step of the process. In order for evaluations to matter, there needs to be a set of standards against which teaching practices can be measured. Currently, many unit policies do not define teaching excellence nor do they provide basic criteria that outline the behaviors or outcomes for meeting minimum teaching quality expectations. In an effort to create clear expectations, the 2017-18 Senate Task Force for Teaching Evaluation drafted a Teaching Evaluation Framework as a starting point. The framework emerged from the data gathered via the tools described above and the UO’s teaching excellence principles. The 2018-19 CAIT and Senate Committee will be engaging with pilot units to adapt this framework to their discipline and pilot it to evaluate faculty’s teaching.
This timeline displays our progress on these initiatives in reverse chronological order.
- January 30 - University Senate approved the CIET senate committee’s work on a Warning and Guidance on Student Evaluations of Teaching document for faculty and personnel committees.
- January 13 - Chronicle of Higher Education article highlights UO’s work on course evaluation.
- Teaching Excellence and Evaluation CAIT Peer Review subgroup members each selects an observation tool to experiment with for Peer Review and completes 3 mock classroom observations.
- Teaching Excellence and Evaluation CAIT Pilot subgroup makes tweaks to the Midterm Student Experience Survey for week 5 and selects End of Term Student Experience Survey to Pilot in week 10. Pilot faculty will also complete Instructor Reflection.
- November 28 - Ginger Clark, assistant vice provost for academic and faculty affairs at the University of Southern California, hosted two discussions about USC's reform of their teaching evaluation process.
- November 17 – Sierra Dawson, Austin Hocker and Lee Rumbarger lead discussion about Teaching Evaluation reform efforts at Professional and Organization Developers Network (POD) conference in Portland, OR.
- Teaching Excellence and Evaluation CAIT Pilot subgroup includes English, Honors College, Human Physiology, Lundquist College of Business, and School of Planning, Public Policy and Management. From these units:
- Teaching Excellence and Evaluation CAIT provided guidance for an update on the definition of Teaching Excellence.
- Teaching Excellence and Evaluation CAIT (Community Accelerating the Impact of Teaching) formed with membership from each and every school and college, as well as the three divisions of the College of Arts and Sciences.
- Continuous Improvement and Evaluation of Teaching Senate Committee formed based on spring Senate Legislation.
- October 3 - University Senate updated with progress report and plan for the year.
- Faculty teaching courses in Business, English, Public Planning, Policy and Management, Writing, Romance Languages, Education Studies, Cinemas Studies, and Human Physiology are piloting the Midterm Student Experience Survey, End-of-Term Student Experience Survey, and 10-Minute Instructor Reflection in the CollegeNET system.
- Pilots of updated Midterm Student Experience Survey in Qualtrics, as well as End-of-Term Student Experience Survey and 10-Minute Instructor Reflection in CollegeNET
- May 1 – Town Hall for undergraduate students, 6 - 7 p.m., EMU Redwood Auditorium
- May 2 – Town Hall for unit heads, 8:30 - 9:30 a.m., EMU Redwood Auditorium
- May 2 – Town Hall for graduate students, 10 – 11 a.m., EMU Redwood Auditorium
- May 3 – Town Hall for faculty, 10 – 11 a.m., Ford Alumni Center Ballroom
- May 9 – Senate discussion of motion
- May 23 –Senate motion US17/18-19 legislates the following:
- The Continuous Improvement and Evaluation of Teaching senate committee will be for
- Student evaluations will be anonymous
- Midterm Student Experience Surveys will become available via CollegeNET
- 10-Minute Instructor Reflections will become available via CollegeNET
- January 10-11 – Site visit from Emily Miller, associate vice president for policy at the Association of American Universities (AAU) – View her presentation Aligning Practice to Policy: Teaching Excellence at AAU Institutions
- February 5 – Visit from Noah Finkelstein, Director of the Center for STEM Learning University of Colorado Boulder to discuss Colorado’s Teaching Quality Framework
- Week 5 – Initial pilot midterm student experience survey in Qualtrics
- January 31 – Report to Senate: Improving How We Evaluate Teaching
- End of term – Initial pilot student experience survey in Qualtrics
- End of term – Initial pilot faculty instructor reflection and graduate instructor reflection in Qualtrics
- March 15 – Senate Notice of Motion: Improving UO’s methods of course evaluation and teaching reviews
- Meetings with stakeholders: Associate Dean’s Luncheon on Teaching Evaluation, Grad Council, ASUO Exec, SWAT, Graduate Student Advisory Board, Women’s Center, Dean of Student Life-Staff, Student Trustee and Senators, Women of Color Coalition, LCB Dean’s Undergraduate Student Advisory Council, Dean of Students Advisory Committee, Mujeres student group, IMPACT (meetings continue through winter and spring terms)
How to Engage
We understand that this process will be a big shift in how the university evaluates teaching, and we want to encourage your feedback and participation as the new tools are developed. You may provide your feedback through the online form below. Be sure to check back for new updates on how you can get involved.
To learn more about the research behind our initiative to revise the UO Teaching Evaluation System, please see the following documents.
- Aligning Practice to Policy: Changing the Culture to Recognize and Reward Teaching at Research Universities
- Teaching, Learning and Achievement: Are Course Evaluations Valid Measure of Instructional Quality at the University of Oregon
- Student Evaluations Can’t Be Used to Assess Professors
- Gender Bias in Teaching Evaluations
- Teaching Evaluations and Bias
- Is Gender Bias an Intended Features of Teaching Evaluations?
- Undergraduate STEM Education Initiative
- The Career Framework for University Teaching
Updated October 2018