Compressive Deep Federated IoT for Privacy Preserving Activity Recognition

Led by Nirmalya Roy, Ph. D.

Nirmalya Roy is currently an Associate Professor and Graduate Program Director in the Department of Information Systems at University of Maryland Baltimore County. He directs the Mobile, Pervasive and Sensor Computing Group (MPSC). He was a Clinical Assistant Professor in the School of Electrical Engineering and Computer Science at Washington State University from January 2012 to June 2013. Prior to that, he worked as a Research Scientist at Institute for Infocomm Research (I2R), Singapore from 2010 to 2011. He was as a postdoctoral fellow in Electrical and Computer Engineering Department at The University of Texas at Austin from 2008 to 2009. He received his Ph.D. and M.S. in Computer Science and Engineering from The University of Texas at Arlington in 2008 and 2004 respectively. He did his Bachelors in Computer Science and Engineering from Jadavpur University, India in 2001.


The goal of this REU project is to develop a Federated learning assisted compressive deep learning based human activity recognition framework which runs on the edge devices in real-time. Recognizing human daily activities in real-time and providing just-in-time feedback on their functional and behavioral health are the most important aspects of making the edge devices (e.g., smart phones, smart wristwatches) real smart and bringing the proactive healthcare in users fingertips. Nevertheless, there is always the risk of privacy breach in this process because in most of these applications unencrypted health related data are fed directly to the base machine learning model. To address this issue, Dr. Roy will guide the student to investigate compressive deep federated and active learning-based approach to build, recognize and learn the human activities while preserving the data privacy and minimizing the resource footprints on edge devices. The federated learning (FL) based framework offers the advantage of automatically updating the main model without direct access to the data in a low-resource setting. The proposed FL assisted distributed activity learning model will be capable of running on emerging Internet- of-Things (IoT) consumer devices and will have significant impact on human functional, behavioral, and physiological health monitoring in real-time. The student will be guided to investigate a state-of-the-art deep learning-based activity recognition model which will be trained on server using public activity datasets. To reduce the computational overheads, we will apply quantization with differing parameter tuning strategies as compression technique before feeding the main model to the edge devices. The edge devices will be trained sequentially on unseen datasets and the trained model on edge will be transported to the server following the push-pull control flow of federated learning. This will help to preserve the privacy of sensitive personalized health related data based on HIPAA act while helping to fine tune the global activity learning model residing in the main server. We plan to benchmark our proposed deep federated learning based model in terms of human activity inference accuracy, precision, recall and memory, power and bandwidth requirements on a variety of resource constraint edge devices with heterogeneous hardware features such as Jetson Nano (GPU), Raspberry Pi with Neural compute stick 2 (NCS2) (VPU), Coral dev board with accelerator (TPU), and iPhone 11 (A13 bionic chip).