Welcome

Objectifying Social Presence: Evaluating Multimodal Degraders in ECAs Using the Heard Text Recall Paradigm

IEEE Transactions on Visualization and Computer Graphics

Embodied conversational agents (ECAs) are key social interaction partners in various virtual reality (VR) applications, with their perceived social presence significantly influencing the quality and effectiveness of user-ECA interactions. This paper investigates the potential of the Heard Text Recall (HTR) paradigm as an indirect objective proxy for evaluating social presence, which is traditionally assessed through subjective questionnaires. To this end, we use the HTR task, which was primarily designed to assess memory performance in listening tasks, in a dual-task paradigm to assess cognitive spare capacity and correlate the latter with subjectively-rated social presence. As a prerequisite for this investigation, we introduce various co-verbal gesture modification techniques and assess their impact on the perceived naturalness of the presenting ECA, a crucial aspect fostering social presence. The main study then explores the applicability of HTR as a proxy for social presence by examining its effectiveness under different multimodal degraders of ECA behavior, including degraded co-verbal gestures, omitted lip synchronization, and the use of synthetic voices. The findings suggest that while HTR shows potential as an objective measure of social presence, its effectiveness is primarily evident in response to substantial changes in ECA behavior. Additionally, the study also highlights the negative effects of synthetic voices and inadequate lip synchronization on perceived social presence, emphasizing the need for careful consideration of these elements in ECA design.

Listening Effort in Complex Audiovisual Scenes Under Varying Simulated Room Acoustic Conditions

Trends in Hearing

Reverberation can degrade speech intelligibility and comprehension, increase cognitive load, and impair memory. Many studies examining reverberation effects use audio-only conditions and stationary background noise. Thus, it remains unclear to what extent findings generalise beyond laboratory setups. In the present study, we examine the influence of reverberation on memory and behavioural listening effort in a complex, audiovisual virtual reality (VR) environment. We implemented a virtual reality (VR) open-plan office with realistic room acoustics and time-varying background noise, including animated virtual agents. Three simulated room-acoustic conditions were compared: a free-field, an acoustically optimised office, and an untreated office. Under these conditions, N = 30 participants performed a dual-task. The primary Heard-Text Recall (HTR) task measured speech comprehension and memory. Participants listened to short stories narrated by two speakers and answered content-related questions. The vibrotactile secondary task measured listening effort. In single-tasking, memory for spoken content was significantly better in free-field than in both offices, with no significant difference between offices. In dual-tasking, vibrotactile response times indicated that listening effort increased in the presence of reverberation. Across single- and dual-task conditions, vibrotactile error rates decreased in the presence of reverberation, possibly due to elevated concentration in more difficult conditions. These findings suggest that room acoustics, compared to free-field environments, influence memory and listening effort in complex audiovisual VR and highlight the importance of incorporating reverberations when designing audiovisual environments for listening research.

How Far is Too Far? The Trade-Off between Selection Distance and Accuracy during Teleportation in Immersive Virtual Reality

IEEE Transactions on Visualization and Computer Graphics

Target-selection-based teleportation is one of the most widely used and researched travel techniques in immersive virtual environments, requiring the user to specify a target location with a selection ray before being transported there. This work explores the influence of the maximum reach of the parabolic selection ray, modeled by different emission velocities of the projectile motion equation, and compares the resulting teleportation performance to a straight ray as the baseline. In a user study with 60 participants, we asked participants to teleport as far as possible while still remaining within accuracy constraints to understand how the theoretical implications of the projectile motion equation apply to a realistic VR use case. We found that a projectile emission velocity of 14 m/s (resulting in a maximal reach of 21.52 m) offered the best trade-off between selection distance and accuracy, with an inferior performance of the straight ray. Our results demonstrate the necessity to carefully set and report the projectile emission velocity in future work, as it was shown to directly influence user-selected distance, selection errors, and controller height during selection.