Investigating Physiological Synchrony from Dyadic Interactions During In-Situ Simulation Training
Led by Andrea Kleinsmith, Ph. D.
Andrea Kleinsmith is an Assistant Professor in the Department of Information Systems. Andrea’s primary research interests are in Human-Computer Interaction and Affective Computing and focus on measuring and modeling body expressions in real world training situations. In one area, she examines human behaviors such as empathy that may be understood from medical students’ interactions with virtual patient training systems. In another area, she models users’ body expressions in order to endow systems with the ability to automatically recognize affect from the body. Andrea’s current aim is to build affect recognition training systems for complex real world situations in the wild with populations such as first responders and other emergency services. Dr. Kleinsmith has held postdoctoral researcher positions in the Virtual Experiences Research Group in the Computer and Information Science and Engineering Department at the University of Florida and in the Embodied AudioVisual Interaction Group at Goldsmiths, University of London in the UK. She received a PhD in Computer Science from University College London, UK, an MSc in Computer Science from the University of Aizu in Japan, and a BA in Psychology from the University of Oregon.
Project:
This project tackles a challenge of Affective Computing; moving from modeling affect at an individual level to understanding and modeling affect at a group level, i.e., within dyads and small groups. The objective of the project is to investigate the role of physiological synchrony on affect and affect-related phenomena within dyadic interactions in real-world scenarios. The extent to which a working pair is physiologically synchronized can shed light on factors affecting team performance, including stress and cognitive load. Research on physiological synchrony (i.e., the unconscious, dynamic linking of physiological responses such as heart rate and electrodermal activity (EDA)) has given us insight into human-human interactions, such as level of social engagement, coordinated behavior, and team performance on digital tasks. We propose to extend this line of work by exploring the physiologic synchrony from EDA data in trainee-trainee and trainer-trainee dyads in high-stakes in-situ simulation training with emergency healthcare providers and surgical trainees. The goal is to better understand the impact of physiological synchrony on team stress, cognitive load and performance. Research has highlighted the need for interventions and systems in training aimed at stress awareness and management. How can we implement and guide affect awareness within teams through collaborative training systems? The intellectual merit of the proposed project lies in modeling physiological synchrony within dyads and developing a comprehensive understanding of its impact on stress and cognitive load. An outcome of the project will be laying the groundwork for collaborative training systems. Adopting the algorithmic approach from existing work, our preliminary results suggest a relationship between the level of physiological synchrony within paramedic trainee dyads and the level of perceived stress. Dr. Kleinsmith will work with REU students to carry out research activities toward this aim. Video and physiological data has already been collected from dyads in two complex medical training situations: paramedic and surgical simulations. The REU students will be mentored on all steps involved in modeling physiological synchrony; processing the EDA data and extracting features relevant for assessing the level of synchrony within dyads, statistically evaluating the level of physiological synchrony, and applying machine learning techniques to build supervised and unsupervised models of synchrony. The REU students will also have the opportunity to assist with other research activities, including video coding, qualitative analysis, study design, and manuscript writing. The proposed activities will give the REU students a comprehensive introduction to the steps involved in conducting research. The students will learn both qualitative and quantitative research methods, software for application of machine learning algorithms, and, software for nonverbal behavior coding. Finally, the students will be mentored through writing up the results of research for publication.