Date of Award

5-2022

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Program

Speech and Hearing Science

Track

Audiology

Research Advisor

Mark Hedrick, PhD

Committee

Aaron T. Buss, PhD Patti Johnstone, PhD Patrick Plyler, PhD Caglar Tas, PhD

Keywords

fNIRS, listening effort, neuroimaging, Pupillometry, semantic information, Speech Recognition

Abstract

Real-world conversations are often accompanied by some sort of interference that challenges the clarity of the speaker’s message, causing listeners to exert more effort to understand speech. Previous research has demonstrated that when listening to speech becomes difficult, various regions of the brain are recruited beyond those which engage during optimal listening conditions. However, the neural correlates that underly listening effort are not fully understood. Importantly, the pupillary response can be used to index listening effort, such that pupil size increases with increasing cognitive demand. I proposed that pupillometry can be used to characterize the cortical response, such that changes in pupil size would be associated with neural activation that directly relates to effortful processing. The primary goal of this study was to investigate the physiological mechanisms underlying recognition of realistic cochlear implant (CI) speech simulations (i.e., vocoded speech) and examine how effort might be alleviated with access to semantic information. To achieve this, I implemented a speech recognition task and manipulated the semantic content of the sentence (Predictability), spectral degradation (Speech Quality), and the way participants reported their response (Task Mode). Concurrent measurements of speech recognition, pupillary responses, and cortical activation via functional near infrared spectroscopy (fNIRS) were recorded from 41 normal hearing (NH) adults. As expected, challenging vocoded speech resulted in larger pupil sizes than did non-vocoded speech, and use of semantic information reduced listening effort. Interestingly, the largest exertion of listening effort was produced after the sentence presentation ended. Neural data revealed Predictability effects in frontal, temporal, and parietal cortices, such that activation in frontal regions responded to higher-level semantic representations of speech, and posterior brain regions were more sensitive to Task Mode which modulated the cortical mechanisms used to resolve Predictability. Positive correlations between the strength of the neural response and peak pupil response were observed in the Middle Frontal Gyrus (MFG) and Inferior Parietal Cortex (IPC), revealing cortical regions related to listening effort. Surprisingly, pupil size just prior to the sentence onset positively predicted individual speech recognition score, suggesting that the degree to which participants were engaged (or prepared) at the start of the trial predicted their overall speech recognition performance. On the other hand, pupil size right after sentence offset was negatively associated individual capacity to exploit semantic information, suggesting that listeners with smaller improvement from Low- to High-Predictability conditions had engaged more effort after the sentence was presented. Overall, the findings presented here indicate that the MFG and IPC carry out cortical mechanisms related to effortful processing, and listeners with a stronger MFG response were more impacted by the degraded listening conditions as they exhibited more effort (larger pupil size) and lower benefit in performance from semantic information. Therefore, it appears that the effect of listening effort on communication (i.e., speech recognition score and semantic capacity) is determined by the individual physiological response of the listener.

Declaration of Authorship

Declaration of Authorship is included in the supplemental files.

ORCID

https://orcid.org/0000-0002-2830-7333

DOI

10.21007/etd.cghs.2022.0594

Additional Files
2022-009-Defenderfer-DOA.pdf (99 kB)
Declaration of Authorship
Download

Share

COinS