Scalable and Adaptable Cross-Domain Autonomous Health Assessment

CAREER Project

The wide availability of commodity smart home sensor systems (Google Home, Amazon Echo, etc.) and internet-of-things (IoT) devices (Fitbit, Actigraph, etc.) is making it easier to continuously monitor individuals' health-related vital signals, activities, and behaviors to provide just-in-time health intervention to the aging population. This CAREER project seeks to design, implement, and evaluate heterogeneous sensor systems in smart homes that help ameliorate the progressive functional and behavioral health decline of older adults. 

In order to realize autonomous health assessment methodologies in practice, it is necessary to build an activity and behavior recognition system across multiple inhabitants and various connected consumer devices that can select, adapt, and cope with device and user heterogeneities, privacy characteristics, resource constraints and scarcity of labeled data. To address the above-mentioned problems, this research project contributes to new methodology in four ways. 

Cross domain activity recognition scenarios: the same set of activities can be performed by different persons and captured using a variety of smart devices across different body positions, introducing a wide range of heterogeneities e.g. device type/sampling rate, personal and positional and any combination of them.

Award Title

CAREER: Scalable and Adaptable Cross-Domain Autonomous Health Assessment

Acknowledgement

This material is based upon work supported by the National Science Foundation under Grant Numbers:  1750936 (University of Maryland Baltimore County)

Disclaimer

Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. 

Duration

May 1, 2018 to April 30, 2025 (Estimated)

Team

Investigators

Graduate Students

Graduated PhD Students

Collaborators

Project Goals

The goals of this project are to develop novel deep transfer learning techniques, optimization-based heuristics, opportunistic sensing architecture, and spatiotemporal dynamical systems-based approaches to address the diversity, adaptability, and reliability of activity and behavior recognition models across different users and technologies, while leveraging a human-in-the-loop control for improving the performance of the sensor systems. 

Research Challenges

The project specifically looks at cross-domain approaches that can accommodate variability in behavior, activity, and physiological health conditions across a large population and diverse set of smart home sensor systems. The inability to build scalable and adaptable activity and behavior monitoring models across domains such as multi-occupant homes with heterogeneous internet-of-things devices is a major impediment to adoption of smart home technologies for healthcare applications. 

Summary of Current Results

Our current project research and development activities are helping to automate activity and physiological health monitoring at scale, and thereby improving the design and study of adaptive interventions for elderly people, their families, and professional caregivers.

AugToAct [13] At a Glance

Semi-supervised learning architecture for scalable human activity recognition introduced in AugToAct [13] leveraging unlabeled data to improve ADL and iADL classification

Accuracy comparison of semi-supervised and augmentation modules in AugToAct[13] with different ratio of labeled and unlabeled samples on two datasets: DAR and HHAR. 

Note: The labeled data percentage is expressed in log scale on the x axis.

AugToAct [13] and HDCNN [18] architectures working in tandem in a semi-supervised transfer learning setup with limited labeled data in both source and target domain


Accuracy comparison of AugToAct[13] and other transfer learning approaches on the DAR dataset. 

Publications


Presentations & Services

Presentation Videos

Presentation on STranGAN: Adversarially-Learnt Spatial Transformer for Scalable Human Activity Recognition at IEEE/ACM CHASE conference on December 16, 2021, Washington D.C., USA

Zaher's Dissertation Defense on Building Robust Human Activity Recognition Models from Unlabeled Data, June 8, 2022

Zahid's invited talk on RhythmEdge: Enabling Contactless Heart Rate Estimation on the Edge at Aberdeen Proving Ground, Army Research Lab, July 2023. This work had received IEEE SmartComp 2022 Best Paper Award.

Datasets

Software Downloads

To be added

Patents

To be added

Broader Impacts

Algorithmic Advances

The project is investigating various state-of-the-art machine learning techniques such as transfer learning, active learning, deep learning with ambient and wearable sensors, IoT technologies and clinical tools in practice to improve the quality-of-life of older adults living independently in retirement community centers, assisted living and smart home environments.

Dataset Advances

Work in progress.

Integration with the broader research community

The PI and his research team have been collaborating with the University of Maryland, School of Nursing for real deployment of smart home sensor systems and technologies at three retirement community centers and senior homes in the greater Baltimore area. 

Educational Material

Dr. Roy has designed a new course on Smart Home Health Analytics and has been offering this course at UMBC every alternative semester. The course has generated many publications related with the broader application of the CAREER project to health and other research areas such as sports analytics, cyber-physical systems and smart city. Here are some publications which have been inspired by the various Smart Home Health Analytics course research projects.

David Welsh and Nirmalya Roy. Smartphone-based Mobile Gunshot Detection, In Proceedings of the 13th Workshop on Context and Activity Modeling and Recognition (CoMoRea’17), co-located with PerCom, March 2017. [pdf] 

H M Sajjad Hossain, Md Abdullah Al Hafiz Khan, and Nirmalya Roy. SoccerMate: A Personal Soccer Attribute Profiler using Wearables, in Proceedings of the 1st IEEE PerCom International Workshop on Behavioral Implications of Contextual Analytics (BICA), co-located with PerCom, March 2017. [pdf] 

Abu Zaher Md Faridee, Sreenivasan Ramasamy Ramamurthy, H M Sajjad Hossain, and Nirmalya Roy. HappyFeet: Recognizing and Assessing Dance on the Floor, in Proceedings of the ACM 19th International Workshop on Mobile Computing Systems and Applications (HotMobile), Feb. 2018 [pdf] 

Varun Mandalapu, Lavanya Elluri and Nirmalya Roy. Developing Machine Learning based Predictive Models for Smart Policing, in Proceedings of the 1st IEEE International Students Workshop on Smart Computing (SmartStudents), co-located with SmartComp, pp. 1-6, Washington D.C., June 2019 

Point of Contact

Last Updated

05/04/2024