Author(s) :

Anda Xie.

Volume/Issue :

Abstract :

Seniors living alone or in nursing homes are often isolated from interpersonal interaction. Existing senior care Intelligent Assistance Technology (IAT) faces challenges, including a rigid conversational structure, a lack of proactive responses, and an inability to address interruptions in a timely fashion. To address these issues, I present a framework that aims to a) develop an Automatic Speech Recognition (ASR) Natural Language Processing (NLP) IAT conversational platform that can parse user speech, analyze speech sentiment, save speech content, and respond with a situationally appropriate tone and content and b) test and implement novel interruption detecting models to simulate authentic conversation. A variety of interruption detection methods were evaluated using the ASR-NLP IAT framework, including facial sentiment analysis, head direction tracking and pupil tracking. Final iterations of this turn-taking technology demonstrator involving facial sentiment analysis reached 84.6% accuracy and an F1 score of 0.6. In conclusion, it is proven that ASR-NLP IAT has matured to the phase where it can effectively simulate person-to-person conversation and fluidly exchange conversational roles.