Creating Room on Zoom: How to Make Your Meetings Accessible for the Hard of Hearing
By: Savvy Gupta: Lead Design Artist, Karina Novotny: Research Coordinator, Tadeusz Pforte: Data Investigator, and Ryan Whitaker: Research and Study Design
What Is at Stake?
Since the start of the COVID-19 pandemic, the shift to remote work has created setbacks for the hard of hearing community in their ability to communicate effectively in the workplace. Poor audio and video quality of other participants in the meeting causes inaccurate live captions as well as the inability to read someone’s lips while they are speaking. Our goal is to make online video conferences as accessible as possible to the hard of hearing community by tackling the problems that are within each participant’s control. By focusing on the problems that are created by an individual’s behavior in an online meeting, we can alleviate some of the difficulties that the hard of hearing community faces while working remotely.
How We Collected Our Data
Our research methods proved to be extremely helpful in determining the focus of our solution. We chose to use a survey that we posted in online communities for those that are hard of hearing, such as on Reddit and AllDeaf. By creating a simple survey with optional elaboration, we were able to elicit many more responses than we were expecting and identify common problems among all of our participants. Our online survey outreach resulted in 22 responses from the hard of hearing and d/Deaf community however the vast majority of extrapolatable data came from those who were specifically only hard of hearing as hearing ability varies widely. All of our participants had experience working remotely via some online platform and most of them experienced issues regarding their hearing ability that negatively affected their virtual meetings.
Additionally, we interviewed a researcher at UW that is currently working on a project involving the hard of hearing community and live captioning services. She has a lot of experience working with this community and was able to guide us towards a different approach to the problem we identified in a way that is both helpful and respectful to the hard hearing community.
What We Found
Through our design research, we were able to isolate a few major themes. Many of our hard of hearing research participants indicated that a large challenge with online video conferencing is the lack of supplemental communication, i.e. difficulties with capturing all of the information in the meeting due to the inability to use multiple forms of communication. For example, hard of hearing participants often rely on auto-captioning and lip-reading to bridge the gap and supplement any information that they missed. However, as many participants noted, and as all users of online video conferencing software can attest to, online meetings are often plagued with audio and video quality issues. Audio quality issues can make it nearly impossible for participants with audio discrimination trouble to understand speakers, and additionally cause accuracy issues in auto-captioning.
An additional piece of our design research was speaking with researchers that have had experience working with and creating technology for the hard of hearing community. Through this, we learned that a more appropriate framework for crafting our design ideas could be to focus our technology on hearing people, instead of hard of hearing people. This prevents the work from falling onto the shoulders of hard of hearing people and instead encourages hearing people to make the virtual environment accessible. This ensures that the message that our design sends to users is not that we are trying to “correct” hard of hearing experiences.
Another theme we found through our research was visual density and dispersion — how crowded visual elements are on the screen and the subsequent eye fatigue that results from having to switch focus from one element to another. This theme came up when participants noted how when captions are available, they can be covered by other elements in the video conferencing software. Customization of visual elements on screen could also be helpful if participants need to enlarge certain participants’ video or screen displays to aid lip-reading/screen reading.
The Design
To us, the most important theme from our research was that we can improve accessibility for hard of hearing and d/Deaf people by improving how hearing people communicate with them, rather than placing a greater onus on hard of hearing people. Many of the issues faced by hard of hearing participants involved external problems out of their control, such as audio or video quality issues and quality of presentation layouts. As a result we decided that it would be best to design for hearing users with the focus of making online meetings more comfortable and accessible to the hard of hearing.
Our design tackles both of the problems stated above in an attempt to both monitor audio and video quality live during a meeting and to analyze presentations beforehand for visual density and dispersion. We decided to focus on both of these tasks because they are equally important for guiding the experience of deaf and hard of hearing conference attendees. Without full reliance on hearing, these users rely more heavily on their sight to both supplementally read captions, lips, and content on the screen. Aiding this visual dependency by assuring smooth audio and video during the meeting means less stress and less friction. Allowing presenters to check the quality of their slide information density additionally helps lessen the burden on the eyes to switch between reading content, captions, and lips. Both of these tasks have the same potential to make consumption of information during presentations smoother for all participants but differ fundamentally in the point of time that they are accomplished, and the specific medium of information (presentations and people).
The first component, presentation analysis, helps presenters create presentations that are easier for hard of hearing (and also hearing) people to keep up with and cause less eye strain. It does this by analyzing the visual density, how much visual information a viewer has to process, including text and complex diagrams, and the visual dispersion, how far apart that visual information is. A fast-paced, visually dense presentation may be difficult for hard of hearing people to keep up with, because they must read both the information on the screen and the captions of the words being said, unlike hearing people who can process the visuals on the screen and the audio from the speaker simultaneously. Similarly, a presentation with high visual dispersion leads to more frequent eye movements and thus greater eye fatigue for all participants, but has a particularly high impact on hard of hearing participants who must also constantly be looking back and forth between the presentation and the captions, which are often placed far away at the bottom of the screen. Our design allows presenters to input their slide deck and receive a simple slide-by-slide rating that helps them easily identify slides that are particularly dense or high in dispersion. Then, they can either improve the slides or know to slow down in critical areas to maximize accessibility.
The second component being behavioral reminders attempts to remedy issues with caption quality and lip reading potential by ensuring smooth quality video and audio throughout a presentation. This component works by reminding a meeting participant or presenter to speak clearly so that all attendees can hear them properly so that closed captioning systems have the best quality auditory data to process accurately. In addition, this component reminds such users to face their camera directly while speaking and to fix any lighting or obstructions that may make their face harder to see. If a hard of hearing meeting participant is unable to properly read lips due to camera quality they will become left behind in the meeting. In this story board you can see different meeting participants encountering alerts that there are specific actions they can take to make the presentation run smoother for everyone else. With such reminders, everyone is able to be sure that they are doing their best to facilitate a smoothly run meeting without putting any burden on the deaf and hard of hearing attendees to speak up about issues or ask for repetitions.
Summary
During our quest to design technology to support the hard-of-hearing and D/deaf community during this transition to work-from-home during the pandemic and beyond, we had the opportunity to learn from a variety of members and researchers of the community. We were able to design a system that would encourage hearing people to put in the effort to make the work environment more comfortable for the hard-of-hearing and D/deaf community rather than putting the burden on these community members. The process of designing tech that incorporates both making presentations more accessible for meetings beforehand and making meetings themselves more accessible through auditory/visual reminders taught us a great deal about planning, designing, and conducting research, and we’re excited to continue working on our idea and bring it to life.
