Using Machine Learning to Understand Student Perspectives on Accessibility: An Analysis of the Student Experience Survey 

Austin Hocker, Charlotte Lloyd, Grant Crider-Phillips, & Sung-Woo Cho

Print-friendly PDF: 

Table of Contents:

1. Exploring Student Comments 
2. Identifying themes in student comments 
3. What specifically about the accessibility of this course helped your learning? 
4. What specific change in the accessibility of the course would help your learning? 
5. Discussion & Recommendations 
6. Appendix A: Top Twenty Distinguishing Words in “Accessibility” Responses* 

The new UO Student Experience Survey (SES) asks open-ended questions about 13 individual teaching practices. For each course, each student selects the single teaching practice they consider most beneficial to their learning and a single teaching practice that needs improvement in order to support their learning, with prompts to leave textual comments about both answer choices. Therefore, student comments represent perspectives on when a teaching practice is either the most salient positive or negative teaching practice for their learning. This report focuses on “ The accessibility of the course, ” one of the teaching practices students can choose and comment on.

To contextualize the student comments related to accessibility, a separate survey question asks all respondents to indicate if the accessibility of the courses is beneficial, neutral, or needing improvement for their learning. Across all SES responses from 2019-2021 (116,331 student responses), accessibility is rated as needing improvement by 3.4% of respondents, as neutral by 16.8% of respondents, and as beneficial for learning by 79.8% of respondents.

Figure 1. Percent of Student Experience Survey comments about teaching practices selected as most beneficial and most in need of improvement.

From the 13 teaching practices, “ The accessibility of the course ” is the least likely to be selected as the most beneficial or most in need of improvement. Accessibility selected as being “most beneficial for my learning” by 1.6% of respondents (1,619 student comments) and is selected as being “most in need of improvement for my learning” by 2.0% of respondents (1,825 student comments). As context, students can only choose the single element that is the most beneficial or most in need of improvement and over 30% of students respond that “none of these elements need improvement.” It is important to keep in mind that the set of responses considered in this analysis come from the small percentage of students who found accessibility to be either the most positive or negative experience relative to other teaching and learning elements. The types and topics of student responses about accessibility might be very different if, for instance, every student was prompted to write about the accessibility of the course.

Though students are least likely to leave comments about accessibility, student comments about accessibility are relatively long among SES comments at 132 characters, roughly 26 words. Compared to student responses to other SES questions, comments about accessibility most frequently mention experiences about Canvas and online . In particular students reference their ability to find, use, or have access to materials like slides and notes . Students also indicate the value of affordable materials, especially expensive, textbooks, and free materials. More commonly used words can be found in Appendix A.

To identify themes from student responses, we used a hybrid approach combining traditional close reading with a statistical text-mining approach called Latent Dirichlet Allocation (LDA),a topic modeling technique that identifies underlying themes in text by looking for latent groupings of features, in our case individual words. We used LDA as an exploratory tool to present an expert researcher with topics of student comments, which the researcher analyzed and synthesized into the five themes presented here. The value of using LDA in this process comes from the neutrality of the algorithm’s topic groupings, which offers the researcher fresh ways of looking at the data that are relatively unconditioned by their prior expectation about what themes are present in student responses. With LDA categorizations as a starting point, the researcher’s human judgment and experience with UO student surveys are then crucial for integrating topic model results into a rich qualitative understanding of student experiences of accessibility.

One student comment concisely represents many of the most common themes students cited as beneficial accessibility teaching practices: 

“Flexible and accommodating teacher, organized Canvas page, digestible reading and course material, punctual with uploading lecture slides and videos.”

The most common themes that emerged from student comments are listed below, along with representative student comments.

1. Canvas organization and accessible course materials 
2. Instructor flexibility and understanding 
3. Ensuring access to student accommodations 
4. Affordability 

1) Canvas organization and accessible course materials

Students cited Canvas course organization and the ease of finding course materials as a key, beneficial teaching practice. Specifically, students referenced how posting lecture recordings supports their learning.

“It was nice to be able to go back to recorded lectures to catch any information I may have missed in real-time. Also, both the professor and the GEs were super accessible, and were always available to chat over email or during office hours, which was very much appreciated.”

“It was super helpful to have both lecture notes and video recordings to refer to after the initial lecture in case students needed help. The accessibility is very thoughtful for other students given crazy time schedules from each individual student.”

“I love that they made the lectures available for us to watch on our own time. This helped me so much because I could watch them when I wanted and at the speed I wanted. I wish it was like this all the time and I love that they understood that the flexibility to do what's best for us is important.”

Students indicated that timely posting of materials was beneficial. For example,

“The Canvas site had all the materials listed, lectures went up same day, and presentation assignment instructions were available.”

“The content being uploaded before class and having class time be dedicated to discussing the content and bring up any concerns was super helpful. ”

“I appreciated that they posted the slides online, not all professors do and it really helps review and going over material I didn't understand.”

Comments suggested making lecture recordings, notes or slides, available allowed for greater student flexibility. For example,

“Everything on Canvas was easy to find and read! The instructor does a great job on uploading lectures and including written explanations of topics (in case you miss a day of class or need to review before an exam). “

“All of the materials are available on his page and are pretty well laid-out, so even if you miss a class due to a connection that's too slow or spotty for Zoom , you aren't missing any content.”

Finally, students indicated the benefit of posting due dates early or regularly to help them plan their schedules.

“The website and clear deadlines and expectations were great. Gave us all the information we needed to know from day 1. Super straight forward and to the point and I enjoyed that very much.”

“The assignments and all the lab slides were accessible through my computer and in advance, so I could feel completely prepared for class regardless of what the rest of my week looked like.”

2) Instructor flexibility and understanding

The flexibility of instructors and their willingness to demonstrate understanding of student barriers was frequently referenced as a beneficial accessibility practice.

“The ability to have life happen where it didn’t dramatically impact my grade was extremely helpful. There was enough flexibility built into the class that I was confident I would be successful.”

“The ability to actually do the work on an accessible timeline is rewarding and helpful. I want to do the work, sometimes other professors assignments overlap. This flexibility really helps.”

“Flexibility is everything for us students. Knowing when we came to class, that you were always coordinating with us on due dates and expectations was wonderful, and I wish more courses functioned this way.”

“The ability to complete each week on my own time was great. Each module was also uploaded on time each week so I did not have to worry about the weekly module not being available when I would have time to dedicate to this course. I really appreciate the instructor providing all the reading materials and videos as free PDFs. It made the course very accessible for all, and the material was extremely relevant/applicable for our lives.”

Specifically, student comments indicated the benefits of allowing for reasonable extensions and deadline flexibility.

“Being able to receive extensions on my assignments was ultimately the determining factor as to whether or not I succeeded in this course.”

“The flexibility with deadlines (given a valid reason) made the class so much better because when I needed to I was able to have more time to make sure I actually understood the readings. Additionally, the instructor treated us like people and recognized that we didn't have the same levels of understanding/experience in the field and adjusted when needed.”

Attendance policies that allow for flexibility were also commonly cited as an accessibility practice.

“The lectures all being recorded allowed for some flexibility in attendance, which was also very helpful.”

“I have, unfortunately, had great difficulty showing up to classes with frequency due to my disability related barriers. The instructor made sure to not dock me points for missing class if it was due to my disability.”

“it was very easy to participate and get points no matter if you could come to class so it was very easy to stay on top of coursework even if issues came up in your personal life.”

“This course was designed well for students with demanding schedules. Having the option for the online track worked really well with my job this term. I was able to complete assignments and lecture content as it fit into my schedule with due dates frequently enough to keep me engaged and hold me accountable.”

Similarly, students frequently cited instructor understanding and their willingness to make changes based on student feedback or concerns.

“How understanding our professor is when things happen in life that prevent us from missing a class or turning in something late.”

“The professor listened to feedback from students if we were having a hard time and changed the class/assignments accordingly.”

“This course is accessible. I mean, I have an issue, I talk to the instructor about it and suddenly it's resolved.”

3) Ensuring access to student accommodations

Students cited the impact of instructors diligently enacting student accessibility accommodations or implementing strategies that support students with disabilities.

“I have accommodations through the AEC, and this instructor is the first instructor in 2 years who actually made an effort to make the classroom accessible to me.”

“The BEST THING about this course was that the instructor allowed us to wear headphones during our exams. As someone with ADHD, I struggle to focus on exams in big lectures, especially while other students are packing up. Being able to cancel out all the noise was extremely beneficial and helped me focus. This likely helped me get an entire letter grade higher on the exams. I am so glad they allowed this and wish other professors did too!”

“I have ADHD and the instructor goes out of their way to provide students with worksheets to help students focus on the most important parts of readings and lectures, helping people who struggle like me stay on track.”

4) Affordability

Finally, some students cite the low-cost of courses as a beneficial accessibility practice.

“All of the materials we needed for this class were free. What was especially helpful was the free textbook we could reference.”

“I spent 100 dollars total on one class just on textbooks but this instructor made sure to not make us buy any and used pdfs instead so that was really amazing!”

The following five themes emerged as the most common student recommendations for improving accessibility:

1. Canvas use, organization, and grade accuracy 
2. Ensuring access for student accommodations 
3. Auditory accessibility 
4. Flexibility, course policies, and assignment timing 
5. Affordability 

1) Canvas use, organization, and grade accuracy

Students wish instructors used Canvas sites to post readings and assignments, that Canvas grades are updated, and that Canvas sites are easy to navigate.

“The use of Canvas would be beneficial to my learning because it would allow me to access material easily.”

“All course materials should be accessible on the designated website for U of O content - Canvas.”

“Posting reading assignments on Canvas rather than in an email we have to search to find.”

“The professor never posted the Canvas page. Getting information on assignments and about the course was akin to pulling teeth.”

Students cite Canvas organization and navigation ease, releasing materials earlier, and transparent due dates and assignment guidelines.

“The Canvas page was not very easy to navigate (AT FIRST). Once I received a walkthrough of the Canvas page, I understood it; however, until then, I struggled with finding all the assignments I needed to complete.”

“The teacher didn’t program Canvas to alert students when the discussions posts were due automatically, this resulted in me and many other students missing the posts completely. My grade was affected by the teacher’s lack of awareness about Canvas.”

“I think having rubrics for the assignments would help a lot. I had trouble figuring out what specifically some assignments needed.”

“Many of the due dates are at odd times, like 6pm instead of midnight, makes it hard sometimes to get done and turned in on time.”

Students also cite the impact of updated and accurate grade-books in Canvas.

“Showing grades on Canvas would be extremely beneficial to the students.”

“Possibly using Canvas, or some system to allow students to see current grades throughout the quarter.”

“I would have appreciated updates on Canvas (grades, project requirements).”

“I had no idea what my grade was all term because there is no Canvas page or other online page. Our schedule also got mixed up but because there was no Canvas page knowing where we were in the syllabus…”

“The necessary assignments are never posted on time, grades weren't released until WEEK 10, assessments were posted weeks late, feedback took forever to be released.”

2) Ensuring access for student accommodations

Students indicated the importance of instructors ensuring access to documented student accommodations.

“There were times in this course that my AEC accommodations were denied.”

“I felt that I was put in a very uncomfortable situation in terms of disclosing my disability related barriers. There were multiple times that I had missed class (almost all of which were because of my disability) and I felt that they didn't quite understand how accommodations work.”

“I just hope that in the future, they can learn more about accommodations so that they can really accommodate for other disabled students. I think it should be mandatory for every instructor to learn about accommodations and it isn't at all exclusive to this instructor. I find it sort of shameful how often disabled students (including me) are not properly accommodated for because of the university's lack of enforcement.”

“I have accommodations with AEC and found it hard to tell my teacher that and even when I did and requested accommodations it was disregarded when it came time to take the exams.”

3) Auditory accessibility

Students frequently comment on strategies to make audio more accessible. For example, captioning videos in class and when posted online.

“I would have loved to see captions on the lectures as this is an online course. As someone with auditory processing issues, I really struggled at times to understand course material that was only presented in audio and with no visible video of the instructors face to aid in understanding.”

“I would have really liked subtitles on the weekly videos and the interview videos. Subtitles are just a really helpful way to keep track of what was said, which can make for better notes and understanding.”

“There were a lot of videos shown by many of the speakers, all of which were lovely videos, but it would be helpful to display subtitles for all the videos for students who are hard of hearing.”

Making sure students can hear the voices in class, or repeating student questions to make sure they are heard.

“The professor does not use a microphone, so it is hard to hear sometimes. It would be very helpful to hear and pay attention if they used a microphone.”

“There should have been a microphone for use by the lecturer. It was difficult to hear at times during lecture which made it harder for me to learn.”

“The communication from this instructor during lectures was difficult for students to hear and understand. When we told him that they were too quiet to hear, they would not adjust his volume enough to be heard.”

“It was almost always impossible to hear other students speak in the classroom for various reasons. However, if the professor were to repeat what students say/the questions they asked so everyone (especially students in the back) could hear. Additionally, when videos are shown, it would be helpful for subtitles to be put on for students who are hard of hearing.”

And making sure that files are compatible with text-to-speech software.

“Just making sure that students have access to a text that is text to speech accessible…”

“In the future it would be helpful to be able to use text to speech on pdf texts (or all texts that are required) because I haven't been able to get a diagnosis of dyslexia, but it's still difficult to keep up with the readings. So, if the all texts could be accessible, even if students don't have school provided accommodations, it would be helpful.”

4) Flexibility, course policies, and assignment timing

Students frequently cite a desire for more flexibility, less strict course policies, and more time on assignments. Multiple students summarized this concern as a differentiation between equity and equality in course policies.

“While I understand and respect the instructor’s emphasis on equality, I do think it’ s important that the professor remain flexible for students. Some, with documentation or not, require classes accommodate differing learning styles, abilities, levels responsibilities, etc. It’s like that equity vs. equality meme/photo... some people need more to be at a vantage point equal to others.”

“The instructor said that if we were to do anything we would do it as an entire class because equality is only fair. As a student with learning disabilities as well as physical disabilities I have to say that this outlook is atrocious. Fairness is making sure that students get the support they need in order to succeed, not to be equal with their peers. I encourage the instructor to look into teaching for all students and not just the able-bodied individuals. As a whole I enjoy their teaching, but this one phrase stuck with me as a point that needs improvement.”

Specifically, students request more time in advance for assignments to allow them flexibility in when they can complete their course work.

“Post readings ahead of time, some students are busy/slow readers and trying to complete large loads of reading in a short time is sometimes impossible.”

“More often than not, assignments and reading materials were posted further into the week which made it exceedingly difficult to get things done on time. Managing my work flow is how I stay on top of things. But I never knew when assignments would be released.”

“I wish the modules would have been released before Monday, especially when the assignment for the week was always due Tuesday. I could have planned my week or if I was busy Monday I never really had any flexibility.”

“Strict deadlines can be difficult to meet for students with disabilities, including depression and challenges with executive functioning. On the other hand, the clarity of expectations (including deadlines) is helpful with planning.”

Students also perceive some course policies to be accessibility barriers – for example,

“I think while it is important for students to complete all the work in due time, I think being able to not complete one assignment would be nice, or like have the worst assignment dropped.”

“It was made very clear at the beginning of the term that there were no circumstances that would make it possible to miss 4 class periods and still pass this class. When my accessibility needs conflicted with this I did not feel comfortable reaching out because this seemed to be a very strictly implemented policy. I understand why attendance would be so important for this type of class, but I would have been willing to go to other sessions or something to make up ones I missed if I felt like this was an option. Overall, the way this policy was presented made me feel like there was nothing I could do to still pass the class, even though I am registered with the AEC, so I just stopped going after I missed 4 class sessions.”

“Having a class that forces students to have their camera on is ablest at worst, annoying and inflexible at best. I dreaded your class because of that. The fact you would call on people who turned their cameras off for a moment was inappropriate.”

5) Affordability

Students indicate providing affordable course materials is a way to make courses more accessible.

“The textbook is irrelevant and too expensive. It's time to end the tyrannical rule of textbooks.”

“The textbook and class materials are extremely expensive and off putting.”

“Although I am very privileged, I know that the course could very easily be too expensive for other students. The textbook and software are pricey, and may become inaccessible for some students. Aside from that, I have very few complaints.”

“The textbook is very expensive and I believe that the class would be more accessible if there was a cheaper textbook available.”

The themes represented above suggest different ways students describe teaching practices that support accessibility. Students interpret accessibility both in terms of accessibility for students with disability-related barriers and in terms of access to course materials and course resources. For example, students describe the importance of instructors ensuring access to accommodations or describe the ways instructors can provide flexibility in assignments, deadlines, attendance, and other course policies. Alternatively, student comments indicate the benefits of making course content and course resources easily accessible and affordable for students.

There is also clear overlap between the themes that emerge from the ways students talk about beneficial accessibility practices and those elements that need improvement. From student comments in response to both open-ended questions, four common themes emerge relating to 1) ensuring access to accommodations, 2) designing for flexibility, 3) Canvas organization and accessible course materials, and 4) affordability. These themes represent the most common themes in student responses, not an exhaustive list of the ways students interpret accessibility.

These common themes also suggest a number of recommendations for instructors aiming to make their courses more accessible. Importantly, even though a small fraction of students select accessibility as the most important teaching practice, many of these recommendations are beneficial for all students. The recommendations below are made in collaboration with the Accessible Education Center and Teaching Engagement Program (TEP). Contact either office for follow up or consultation ( AEC or TEP ).

1. Ensuring access to accommodations

Instructors can familiarize themselves with their responsibility and best practices to ensure access to documented student accommodations. Students who have Accessible Education Center (AEC) accommodations may choose to communicate with their instructors about their accommodations and instructors are required to ensure access to those accommodations or work with AEC to engage in an interactive process that explores possible accommodations and how to implement them. Students may choose not to request accommodations, particularly if the course, materials, and teaching style are adequately accessible as is. Some key recommendations for instructors include clearly defining course objectives and learning goals, clearly articulating how each course element supports these objectives and goals, and then considering a variety of ways that these objectives and goals might be met and assessed. Doing so will help guide any accommodations related conversations while also providing a more accessible learning environment that leverages Universal Design for learning (UDL) practices. Instructors can familiarize themselves with how to design learning goals using a UDL perspective here .

2. Designing for flexibility

Instructors can implement approaches like UDL, which provides flexibility and reduces barriers for students. This approach can include designing policies, assignments, and documents for all users, rather than the average user. Instructors can learn more about universal design for learning and flexible teaching approaches at the AEC website as well as UDL in course design at CAST’s UDLonCampus website .

Instructors can maintain an updated Canvas site with class recordings, slides, or lecture notes which provides more flexible access to content if they aren’t able to participate in a class session. Additionally, instructors can communicate with students about alternatives to participation in class and specify policies for attendance in the course syllabus .

For uploaded text documents, use options that are text-to-speech readable . For more in depth guides for creating accessible documents, please see webaim’s website .

In lectures, instructors can use a microphone (even if the room is small) to help more students hear you. If instructors post videos or lecture recordings, be sure to include captioning .

To allow student scheduling flexibility, instructors can post assignments as early as possible to have them available for longer periods. Many students work, or have other responsibilities that make short turn-arounds more difficult.

3. Canvas organization and accessible course materials

Instructors should use Canvas for all courses with more than five students (per UO senate legislation ). Consider using Canvas to post essential course information like readings or assignments and set due dates so that assignments show up in student’s Canvas calendars. Students indicate the benefits of consistently organized materials posted at regular time intervals. Consider using an intuitive and descriptive naming convention (such as “UO101 week 01 reading 1 of 3 – Welcome”) for any files used in Canvas and refer to them by this same file name in the syllabus, reading schedule, and throughout Canvas. To help students find class materials, consider walking students through the Canvas course site or telling students exactly where to find the needed Canvas materials. Students appreciate access to notes, slides, and lecture recordings, both for flexibility, and for studying and reviewing content.

4. Affordability

Instructors should consider open educational resources (OER) or other low-cost course materials. Instructors interested in OERs for their course can learn more here or contact oer@uoregon.edu .

Other student comments on accessibility

Student comments about accessibility in the classroom are not limited to the two open-ended responses specifically linked to the selection of “accessibility” as either the most positive or negative learning element. In fact, over 1500 additional comments mention accessibility, accommodations, or AEC as a response to other questions. Comments about “support from the instructor” frequently reference making accommodations , and AEC, or reference how accessible the instructor was. Additionally, over 200 comments about “course materials” and “organization” reference accessibility, primarily related to easily finding course materials.

Differences by class size

Larger classes (over 50 students) are more likely to receive more comments about the accessibility of the course being beneficial for their learning. However, class size does not impact the percentage of students commenting on accessibility needing improvement.

Impact of COVID

The shift in teaching and the challenges of the pandemic influenced student responses to the SES. The SES was launched in Fall of 2019 but was on pause during Spring 2020 due to the pandemic. Of the five terms of data included in this analysis, classes during the three terms of the 2020-21 academic year were significantly impacted by COVID. During the pandemic, students were more likely to select “accessibility” as the most beneficial (1.4% in 2019-20, 1.9% in 2020-21) and as the most in need of improvement (1.9% in 2019-20, 2.1% in 2020-21) suggesting “accessibility” was slightly more salient than other elements of their courses during COVID.

Words having to do only with technology are underlined. Words that the authors thought were interesting or actionable are bolded. The main groups of words have to do with technology , cost , and accommodation .

*Calculated by comparing the tf-idf of words in answers to accessibility questions to those in answers to other questions. More information is available in an upcoming methods paper.