[Colloq] Thesis Defense: Embodied Agents for Long-Term Interaction

[Jessica Biron]

The College of Computer and Information Science presents a PhD thesis defense: 

Speaker: Daniel Schulman 

Date: Thursday December 20th at 2:00pm 
Location: 366 WVH 

Abstract: 
An Embodied Conversation Agent (ECA) is a computer interface designed to simulate human face-to-face conversation with its users, through the production of synthesized or prerecorded speech combined with an humanoid embodiment: a representation, either physical or visual, of a body capable of producing some portion of the nonverbal behaviors associated with speech, such as mouth movements, eye movements, head movements, hand gestures, facial expressions, and body posture. A key research problem in the design and implementation of realistic ECAs is generating the range of verbal and nonverbal behavior present in human conversation with appropriate frequency, timing, and quality. 

ECAs have been used in a variety of applications, motivated by their potential to leverage the affordances of face-to-face conversation to build trust and engagement with users and their learnability given their use of universally understood communicative cues. Many such applications, including education and counseling, are examples of long-term interaction, in which an effective agent must have many conversations, over a long period of time, while building rapport with its users. However, prior work on realistic conversational behavior for ECAs has focused heavily on single conversations, isolated from any larger context. To the extent that human conversational behavior is not fixed and unchanging across multiple conversations with the same conversation partner, this approach risks producing ECAs with behavior that becomes increasingly unrealistic in long-term interaction. 

In this thesis, I present an approach to designing ECAs with realistic verbal and nonverbal behavior in long-term interaction. Based on a longitudinal corpus of health behavior change counseling dialogue, containing multiple conversations between each of several 
counselor-client dyads, I construct a series of statistical models demonstrating systematic changes in human conversational behavior across multiple conversations; these changes are predicted both by the interaction history of a dyad, and by the strength or quality of their interpersonal relationship. Based on these findings, I present a model and implementation of verbal and nonverbal behavior generation for ECAs which reproduces some of the observed behavior patterns. Finally, I present a longitudinal randomized controlled evaluation study demonstrating that the resulting model of behavior generation, implemented in an ECA that acts as a virtual health behavior change counselor, produces measurable improvements in user-agent interpersonal bond in long-term interaction. 

COMMITTEE: 
Timothy Bickmore (advisor) 
Harriet Fell 
Matthew Goodwin 
Linda Tickle-Degnen (external, Tufts) 

Jessica Biron 
Administrative Assistant – Office of the Dean and CCIS Development 
College of Computer and Information Science 
Northeastern University 
202 West Village H 
617-373-5204 
bironje at ccis.neu.edu 
http://www.ccs.neu.edu/