Revising UO’s Teaching Evaluations

Student Feedback
Self Reflection
Peer Review
Documents
Course Surveys
Feedback
Timeline
Evaluation
References

The Office of the Provost and the University Senate have been working together since spring 2017 to revise University of Oregon’s teaching evaluation system. Research indicates that student ratings may not accurately reflect teaching quality and may be inflected by bias [3, 4, 5, 6, 9, 10]. A research project at the University of Oregon similarly cast doubt on the reliability of numerical course evaluations [2].

The Association of American Universities (AAU) and other universities around the globe from University of Colorado, Boulder to University College London, England have argued that it is time for universities’ ideals regarding teaching excellence to align with their policies [1, 7, 8]. As such, the University of Oregon has developed a holistic new teaching evaluation system that does more than simply replace problematic evaluation instruments. The new system provides the path to define, develop, evaluate, and reward teaching excellence. The goals of the new system are to ensure teaching evaluation is fair and transparent, is conducted against criteria aligned with the unit’s definition of teaching excellence, and includes input from students, peers and the faculty themselves.

The Senate Continuous Improvement and Evaluation of Teaching (CIET) committee (established in April 2019 legislation), oversees implementation of Senate legislation related to teaching evaluation. 

COURSE SURVEYS

1. Midway Student Experience Survey (M-SES)
Term: Week 4 | Semester: Week 7
Opens: 8 a.m., Monday
Closes: 6 p.m. Friday

2. End-of-course Student Experience Survey (E-SES)
Term: Week 9-10 | Semester: Week 15
Opens: 8 a.m., Wednesday
Closes: 8 a.m. Monday
*Replaces previous Course Evaluations

3. Instructor Reflection
Opens: 8 a.m., Monday, of Week 10 for terms and Week 15 for semesters.
Closes: 6 p.m. Friday of the first week of the following term or semester.

THE CONTINUOUS IMPROVEMENT AND EVALUATION OF THE TEACHING SYSTEM

There are three important inputs included in the evaluation of teaching system. These include student feedback, instructor reflection, and peer review. New tools are available in each of these categories to support quality teaching and ensure that instructors receive useful and constructive feedback.

Student Feedback

Student feedback is an important part of the Continuous Improvement and Evaluation of Teaching System because students are at the center of the teaching and learning endeavor and have valuable perspectives to share. Students also have the potential to provide instructors with useful feedback to clarify which teaching practices are most beneficial to their learning experience, and what could be improved to help them learn. 

Midway Student Experience Survey (M-SES)
The University Senate approved a centrally distributed Midway Student Experience Survey (M-SES) in which student feedback is only available to the instructor and is intended for mid-course adjustments or for clarifying goals and expectations with students. Campus-wide distribution began fall 2019. 

Students are invited to complete the M-SES during week 4 of the term (or week 7 for the School of Law semester) between Monday at 8 a.m. and Friday at 6 p.m. In order to improve response rates and receive high-quality feedback, instructors are encouraged to provide 10-minutes of class time for students to take the survey. The results of the survey will be available to instructors the following week on Monday at noon; instructors are encouraged to debrief with their students about the feedback they received.

Instructors who have their own midway student feedback instruments and do not plan to read student feedback from the M-SES should communicate this clearly to students prior to week 4 of the term (or law semester week 7) so students know to skip that specific course when completing their M-SES surveys.

End-of-course Student Experience Survey (E-SES)
The University Senate also approved a new End-of-course Student Experience Survey (E-SES) to replace the previous student Course Evaluation. After a year of pilot testing and modifications, campus-wide deployment began fall 2019. The E-SES is focused on student learning, asks concrete questions about specific teaching practices, and inquires about student contributions to their own learning. In reports, student comments are collated by teaching practice making it easier to digest and interpret student feedback.

Analyses from E-SES pilots indicated this survey increases the number of student comments by 60% and drastically reduces the frequency of personal comments (from 21% to 1.5%). Additionally, most E-SES comments (61%), are positive, as students elaborate on teaching practices that are beneficial for their learning.

The E-SES will be available for students the last week of the term or semester prior to finals (Week 10 for the term, or Week 15 for the School of Law semester) from Monday at 8 a.m. to Friday at 6 p.m. For best response rates and high-quality feedback, instructors are encouraged to use the in-class protocol and provide 10-minutes of class time during Week 10/Week 15 for the students to complete their E-SES. Instructors can view reports the Wednesday after grades are submitted.

Implemented Fall 2019, the CIET senate committee created a process to remove discriminatory, obscene, or demeaning SES comments committee as described by the protocol. Instructors can “flag” comments for possible redaction which will be reviewed by the CIET senate committee at least three times per year between October and May, and the upcoming review date will be listed on the main Course Surveys page in DuckWeb.

Instructors should view their feedback on UO’s Instructor Dashboard. 

Mock E-SES

Self Reflection

A crucial part of teaching evaluation is an instructor’s reflection of their own teaching. The UO believes self-reflection should be a regular part of an instructor’s course planning and development, and that the instructor’s own voice should be available at the course level to complement student feedback from the E-SES.

To facilitate on-going self-reflection, the University Senate approved a new Instructor Reflection, which was deployed campus-wide fall 2019 and updated in 2024 to shorten and simplify the form. The tool provides an opportunity for instructors to archive what went well and what might be improved in the future as well as how their teaching aligns with UO’s definition of teaching excellence. The reflection also provides a new mechanism for the instructor’s own voice to inform evaluators’ interpretation of student feedback.

Peer Review

Peer review of teaching at the University of Oregon is a faculty peer's written assessment of how an instructor enacts professional, inclusive, engaged, and research-informed teaching (and other standards that are part of a unit’s Teaching Evaluation Rubric) based on, for example, a class observation, contextual materials like the syllabus and Canvas site, a conversation between the instructor and the reviewer, and an instructor’s answer to standard questions devised by the unit. 

The Senate approved legislation in winter 2023, laying out guidelines for the substance and management of peer reviews of teaching; units then submitted their peer review policies to their deans and the Office of the Provost in spring 2024. 

TEP provides resources to support faculty colleagues in giving substantive constructive feedback. 

EVALUATION OF TEACHING CRITERIA

Revising UO’s evaluation tools is only one step of the process. In order for evaluations to matter, there needs to be a set of standards against which teaching practices can be measured. Teaching at UO will meet expectations when it is professional, inclusive, engaged and research-informed, as described on the University of Oregon Teaching Evaluation Standards, along with any additional standards adopted by units into their policies.   

As part of their Career Faculty and Tenure-Related Faculty Review and Promotion policies, all units submit a Teaching Evaluation Rubric and define what it means to meet, exceed, and fall below expectations related to these standards. (See the Office of the Provost’s AY24-25 Unit Policy Review Process page).  

FEEDBACK

The Continuous Improvement and Evaluation of Teaching (CIET) senate committee and the Office of the Provost are interested in feedback from all stakeholders in this process, including students, faculty, staff and administrators. Use this Qualtrics form to provide your feedback on surveys, reports or P&T related webpages and templates which will be reviewed each term. You must log-in with your duck-id and password to access the Qualtrics form.

Give Feedback

TIMELINE

This timeline displays our progress on these initiatives in reverse chronological order.

Spring 2024

Winter 2023

  • UO Senate Legislation on peer review of teaching to improve alignment to teaching standards. Academic units can update their own peer review policies (examples).
  • Implement Instructor added questions: Instructors can add two custom questions to their student experience surveys. 
  • UO awarded $500k Inclusive Excellence grant to advance evaluation of inclusive and effective teaching.

Fall 2022

  • CIET analyzes and removes questions from end-of-course and midway SES and improves user experience to help improve response rates. 

Spring 2022

Winter 2022

Fall 2021

  • SES feedback window increased to 12 days (Wednesday of week 9 to Monday of finals week)

Spring 2021

Summer 2020 terms only

  • COVID Pandemic - SES results are only available to instructors and instructors can opt-out of SES entirely. 

Spring 2020 terms only

  • COVID Pandemic - SES paused for all instructors

Winter 2020

  • United Academics and University sign an amendment to the August 2019 Memorandum of Understanding (MOU) providing units until Fall 2021 to make unit specific modifications to the standards described in the original MOU. Note that the standards and conditions outlined in the August 2019 MOU will still go into effect as the university’s baseline definition of teaching quality in Fall 2020.

Winter Meetings with UO Faculty and Leadership:

  • January – Department of Biology, faculty meeting
  • January – Department of Creative Writing, faculty meeting
  • January – Department of Political Science, faculty meeting
  • February – Department of Romance Languages, faculty meeting
  • February – Department of Physics, faculty meeting
  • February - All Unit Heads Meeting - Winter Timely Updates
  • February - Senior Leadership Team - Vice Presidents
  • March - Faculty Advisory Council
  • March - Lundquist College of Business - optional faculty meeting

Fall 2019

  • Campus-wide deployment of:
  • Continuous Improvement and Evaluation of Teaching (CIET) Senate Committee develops protocol for redaction of discriminatory, obscene or demeaning E-SES comments
  • 2019/20 Teaching Excellence and Evaluation Community Accelerating the Impact of Teaching (CAIT) group formed with representation from all schools and colleges with a focus on the Teaching Evaluation Criteria document and new data dashboard.
Fall Meetings with UO Faculty and Leadership
  • August - College of Education, leadership meeting
  • August – School of Law, faculty meeting
  • September - Summit for Academic Leaders
  • September – School of Music and Dance, faculty meeting
  • September – School of Architecture and the Environment, faculty meeting
  • September – Department of International Studies, faculty meeting
  • September – Professional Schools Associate Deans meeting
  • October – Department of Women, Gender & Sexuality Studies, faculty meeting
  • October – School of Journalism and Communication, faculty personal committee meeting
  • October – Department of Special Education, faculty meeting
  • October – Department of Philosophy, committee meeting
  • October – Portland Campus, zoom meeting for faculty
  • November – Department of Art, faculty meeting
  • November – Department of East Asian Languages & Literatures, faculty meeting
  • November – Department of Sociology, faculty meeting
  • November – Department of Dance, faculty meeting
  • November – School of Planning, Public Policy and Management, faculty meeting
  • November – Division of Humanities, heads & managers meeting
  • November – Getting More from Peer Review workshop, for faculty and heads
  • December – ‘First Cohort’ workshop, faculty up for review under new standards Fall 2020
  • December – Department of History, faculty meeting
  • December – Department of Educational Methodology, Policy and Leadership, faculty meeting
  • December – History of Art and Architecture, faculty meeting
  • December – Department of Comparative Literature, faculty meeting

Summer 2019

  • United Academics and University sign a Memorandum of Understanding (MOU) enshrining professional, inclusive, engaged and research-informed teaching standards and baseline conditions for use in formal teaching evaluation.  

Spring 2019

Winter 2019

  • March 20 - Updates on progress shared at All Unit Heads training.
  • February 18 - Around the O: Workplace story outlining reform efforts.
  • Teaching Excellence and Evaluation CAIT Pilot subgroup continues to pilot new tools:
  • 1107 students complete the most recent iteration of the Midterm Student Experience Survey for 171 courses
  • Revised End-of-Term Student Experience Survey for Winter 2019
  • January 30 - University Senate approved the CIET senate committee’s work on a Warning and Guidance on Student Evaluations of Teaching document for faculty and personnel committees.
  • January 13 - Chronicle of Higher Education article highlights UO’s work on course evaluation.
  • Teaching Excellence and Evaluation CAIT Peer Review subgroup members each selects an observation tool to experiment with for Peer Review and completes 3 mock classroom observations.
  • Teaching Excellence and Evaluation CAIT Pilot subgroup makes tweaks to the Midterm Student Experience Survey for week 5 and selects End of Term Student Experience Survey to Pilot in week 10. Pilot faculty will also complete Instructor Reflection.

Fall 2018

  • November 28 - Ginger Clark, assistant vice provost for academic and faculty affairs at the University of Southern California, hosted two discussions about USC's reform of their teaching evaluation process.
  • November 17 - Sierra Dawson, Austin Hocker, and Lee Rumbarger lead discussion about Teaching Evaluation reform efforts at Professional and Organization Developers Network (POD) conference in Portland, OR.
  • Teaching Excellence and Evaluation CAIT Pilot subgroup includes English, Honors College, Human Physiology, Lundquist College of Business, and School of Planning, Public Policy and Management. From these units:
  •  
  • Teaching Excellence and Evaluation CAIT provided guidance for an update on the definition of Teaching Excellence.
  • Teaching Excellence and Evaluation CAIT (Community Accelerating the Impact of Teaching) formed with membership from each and every school and college, as well as the three divisions of the College of Arts and Sciences.
  • Continuous Improvement and Evaluation of Teaching Senate Committee formed based on spring Senate Legislation.
  • October 3 - University Senate updated with progress report and plan for the year.

Summer 2018

  • Faculty teaching courses in Business, English, Public Planning, Policy and Management, Writing, Romance Languages, Education Studies, Cinemas Studies, and Human Physiology are piloting the Midterm Student Experience Survey, End-of-Term Student Experience Survey, and 10-Minute Instructor Reflection in the CollegeNET system.

Spring 2018

  • Pilots of updated Midterm Student Experience Survey in Qualtrics, as well as End-of-Term Student Experience Survey and 10-Minute Instructor Reflection in CollegeNET
  • May 1 - Town Hall for undergraduate students, 6:00-7:00 p.m., EMU Redwood Auditorium
  • May 2 - Town Hall for unit heads, 8:30-9:30 a.m., EMU Redwood Auditorium
  • May 2 - Town Hall for graduate students, 10:00-11:00 a.m., EMU Redwood Auditorium
  • May 3 - Town Hall for faculty, 10:00-11:00 a.m., Ford Alumni Center Ballroom
  • May 9 - Senate discussion of motion
  • May 23 –Senate motion US17/18-19 legislates the following:
    • The Continuous Improvement and Evaluation of Teaching senate committee will be for
    • Student evaluations will be anonymous
    • Midterm Student Experience Surveys will become available via CollegeNET
    • 10-Minute Instructor Reflections will become available via CollegeNET

Winter 2018

Fall 2017

  • Meetings with stakeholders: Associate Dean’s Luncheon on Teaching Evaluation, Grad Council, ASUO Exec, SWAT, Graduate Student Advisory Board, Women’s Center, Dean of Student Life-Staff, Student Trustee and Senators, Women of Color Coalition, LCB Dean’s Undergraduate Student Advisory Council, Dean of Students Advisory Committee, Mujeres student group, IMPACT (meetings continue through winter and spring terms)

Spring 2017

DOCUMENTS
REFERENCES

To learn more about the research behind our initiative to revise the UO Teaching Evaluation System, please see the following documents.

  1. Aligning Practice to Policy: Changing the Culture to Recognize and Reward Teaching at Research Universities
  2. Teaching, Learning and Achievement: Are Course Evaluations Valid Measure of Instructional Quality at the University of Oregon
  3. Student Evaluations Can’t Be Used to Assess Professors
  4. Gender Bias in Teaching Evaluations
  5. Teaching Evaluations and Bias
  6. Is Gender Bias an Intended Features of Teaching Evaluations?
  7. Undergraduate STEM Education Initiative
  8. The Career Framework for University Teaching

  9. Boring, A. (2017). Gender biases in student evaluations of teaching. Journal of Public Economics, 145, 27. DOI: 10.1016/j.jpubeco.2016.11.006 at https://www.sciencedirect.com/science/article/abs/pii/S0047272716301591

  10. MacNell, L., Driscoll, A. & Hunt, A.N. (2015). What’s in a name: Exposing gender bias in student ratings of teaching. Innovative Higher Education, 40, 291. DOI: 10.1007/s10755-014-9313-4

Updated January 2020