Led by Nirmalya Roy, Ph. D.

Nirmalya Roy is currently an Associate Professor and Graduate Program Director in the Department of Information Systems at University of Maryland Baltimore County. He directs the Mobile, Pervasive and Sensor Computing Group (MPSC). He was a Clinical Assistant Professor in the School of Electrical Engineering and Computer Science at Washington State University from January 2012 to June 2013. Prior to that, he worked as a Research Scientist at Institute for Infocomm Research (I2R), Singapore from 2010 to 2011. He was as a postdoctoral fellow in Electrical and Computer Engineering Department at The University of Texas at Austin from 2008 to 2009. He received his Ph.D. and M.S. in Computer Science and Engineering from The University of Texas at Arlington in 2008 and 2004 respectively. He did his Bachelors in Computer Science and Engineering from Jadavpur University, India in 2001.


There are various smart home management technologies available nowadays such as smart thermostats, garage door openers, smart lights, Wi-Fi plugs etc. The wide proliferation and availability of smart home health technologies help promote the independence of older adults in their living environments. However, these technologies are not always customized properly to fulfill the needs of older adults with cognitive impairment and physical disabilities. For example, smart thermostats or smart lights could be controlled by an App but most of the times, the older adults are not well engaged in using smart phone apps or have cognitive decline to appropriately navigate through the different digital options they may have. To address these critical issues, we plan to first integrate the voice control agent (Alexa) with smart thermostat and smart light system. The development of such an integrated system with Alexa and smart thermostat or smart light will help posit several technical and design challenges. We plan to first quantify the easiness and acceptability of use of Alexa voice command interface among older adults. Does Alexa require a complex set of spoken instructions to set the appropriate settings and schedules on the smart devices or does it ask successive questions to refine the query to make it easier for older adults? Can it support ample delay in these query replies? More specifically, do the older adults feel comfortable in interfacing and successfully communicating with Alexa voice interface when controlling their smart devices? Alternatively, we plan to investigate the information and control flow pipeline between Alexa and smart devices. Are these smart devices able to set the desired settings properly after Alexa have successfully translated the voice commands to device specific instructions? How much control can Alexa provide over the devices compared to their digital menu/app driven counterpart? Are there any override controls over these devices via remote app for the primary caregivers in case of wrongly set parameters (e.g., wrong temperature schedule which can be life threatening in some cases) through device interface or Alexa? Can these Alexa + smart device setups sustain amicable performance when there is temporary internet blackout? To address these research questions, we plan to investigate the following. (i) bidirectional flow of information from Alexa to smart home sensor systems and its actuation and control in real-time, (ii) successful execution of the voice commands from the older adults based on natural language processing techniques, (iii) heterogeneity of the interactions across different older adults including male and female with the identical device setup (Alexa + smart thermostat, and Alexa + smart light), (iv) engagement, adherence and accountability of the older adults population with this voice agent augmented smart home sensor system. In this project, the REU students will deploy some of these commercially available off-the-shelf smart home technologies such as smart thermostats and smart lights and will find out how the older adults are interacting and engaging themselves with these technologies. Oftentimes, these standalone technologies need manual intervention to control and actuate them. We will link the voice control agent with the smart thermostat to help older adults to control the room temperature from anywhere. Similarly, we will explore how the voice control agent could help to switch off the smart light, for example in downstairs without requiring any movements from the older adults. In addition, the REU students will explore the use of multiple Alexa enabled devices such as Echo in each room and experiment to find the optimal positioning of the devices so that the requests from the older adults are recorded by Alexa. Finally, they will be guided to compare the response of the older adults about using one Alexa enabled device and multiple such devices and the reasons for choosing them.