Course Descriptions: This course will examine different machine learning methodologies, toolsets, wearable and ambient systems that are warranted to recognize, discover, and learn the human activities, behaviors, and their profound impact on human health and wellness in smart home environments. Human activity recognition is a growing field of research with its far-reaching impact on proactive health care, physical fitness, and healthy living. We will discuss different machine learning algorithms, evaluation metrics, and their application to human activity and behavior recognition. A special emphasize will be given to the techniques with respect to a variety of emerging smart environment contexts spanning across monitoring activities of daily living (ADLs), physiological signals (heart rate, galvanic skin response) and psychological behaviors (emotion, stress, depression, agitation) etc. These human signals can be sensed and measured using wearable and IoT (Internet-of-Things) technologies such as smart phones and wristwatches to keep people informed about their activity, life-style, health, mood, behavior, and surrounding. Students are required to do a group or individual project. The tools necessary to develop the project will be reviewed and made available in class, but students are expected to be comfortable with basic use of software, smart phones and smart watches and have an interest in using technology for smart home health monitoring, assessment, and intervention.

Course Topics:

• Gerontechnology

• Supervised Learning

• Bayesian Decision Theory

• Parametric Methods

• Decision Theory

• Machine Learning Evaluation

• Machine Learning Toolkits

• Activity Learning

• Smart Home Health Technologies

• Functional and Behavioral Health Assessment

• Qualitative and Quantitative Clinical Health Assessment Tools

• Applications

In this course we will be discussing various papers recently published in ACM/IEEE MobiSys, SenSys, BuildSys, PerCom 2019. Please check the following papers.

Smartphone Sensing

VitaMon: Measuring Heart Rate Variability Using Smartphone Front Camera; Sinh Huynh (Singapore Management University); Rajesh Balan (Singapore Management University); JeongGil Ko (Yonsei University); Youngki Lee (Seoul National University); ACM SenSys 2019 Zahid Hasan

SenseHAR: A Robust Virtual Activity Sensor for Smartphones and Wearables; Jeya Vikranth Jeyakumar (University of California, Los Angeles); Lianghzen Lai (ARM); Naveen Suda (ARM); Mani Srivastava (University of California, Los Angeles); ACM SenSys 2019 Jason Jozwiak

ALT: Towards Automating Driver License Testing using Smartphones; Akshay Nambi (Microsoft Research); Ishit Mehta (Microsoft Research); Anurag Ghosh (Microsoft Research); Vijay Lingam (Microsoft Research); Venkat Padmanabhan (Microsoft Research); ACM SenSys 2019 Sabrina Mamtaz Nourin

HyperSight: Boosting Distant 3D Vision on a Single Dual-camera Smartphone; Zifan Liu (Shanghai Jiao Tong University); Hongzi Zhu (Shanghai Jiao Tong University); Junchi Chen (Shanghai Jiao Tong University); Shan Chang (Donghua University); Lili Qiu (University of Texas at Austin); ACM SenSys 2019

Deep Learning

MetaSense: Few-Shot Adaptation to Untrained Conditions in Deep Mobile Sensing; Taesik Gong (KAIST); Yeonsu Kim (KAIST); Jinwoo Shin (KAIST); Sung-Ju Lee (KAIST); ACM SenSys 2019

Moving Target Defense for Embedded Deep Visual Sensing against Adversarial Examples; Qun Song (Nanyang Technological University); Zhenyu Yan (Nanyang Technological University); Rui Tan (Nanyang Technological University); ACM SenSys 2019

Neuro.ZERO: A Zero-Energy Neural Network Accelerator for Embedded Sensing and Inference Systems;Seulki Lee (University of North Carolina at Chapel Hill); Shahriar Nirjon (University of North Carolina at Chapel Hill); ACM SenSys 2019

DeepAPP: A Deep Reinforcement Learning Framework for Mobile Application Usage Prediction; Zhihao Shen (Xi'an Jiaotong University); Kang Yang (Xi'an Jiaotong University); Wan Du (University of California, Merced); Xi Zhao (Xi'an Jiaotong University); Jianhua Zou (Xi'an Jiaotong University); ACM SenSys 2019 Millie Zeng

Infrastructure Sensing

WISDOM: Watering Intelligently at Scale with Distributed Optimization and Modeling; Daniel A. Winkler (University of California, Merced); Alberto E. Cerpa (University of California, Merced); ACM SenSys 2019 Hemlata Kohin

Caesar: Cross-camera Complex Activity Recognition; Xiaochen Liu (University of Southern California); Pradipta Ghosh (University of Southern California); Oytun Ulutan (University of California, Santa Barbara); B.S. Manjunath (University of California, Santa Barbara); Kevin Chan (ARL); Ramesh Govindan (University of Southern California); ACM SenSys 2019

SmrtFridge: IoT-based, User Interaction-Driven Food Item & Quantity Sensing; Amit Sharma (Singapore Management University); Archan Misra (Singapore Management University); Vengateswaran Subramaniam (Singapore Management University); Youngki Lee (Seoul National University); ACM SenSys 2019 Prachi Mardhekar

WideSee: Towards Wide-Area Contactless Wireless Sensing; Lili Chen (Northwest University); Jie Xiong (UMass Amherst); Xiaojiang Chen (Northwest University); Sunghoon Ivan Lee (UMass Amherst); Kai Chen (Northwest University); Dianhe Han (Northwest University); Dingyi Fang (Northwest University); Zhanyong Tang (Northwest University); Zheng Wang (Lancaster University); ACM SenSys 2019

A Closer Look at Quality-Aware Runtime Assessment of Sensing Models in Multi-Device Environments; Chulhong Min (Nokia Bell Labs); Alessandro Montanari (Nokia Bell Labs); Akhil Mathur (Nokia Bell Labs and University College London); Fahim Kawsar (Nokia Bell Labs and TU Delft); ACM SenSys 2019

Smart Cities

• One Size Does Not Fit All: Multi-Scale, Cascaded RNNs for Radar Classification; Dhrubojyoti Roy, Sangeeta Srivastava (The Ohio State University); Aditya Kusupati (University of Washington, Microsoft Research India); Pranshu Jain (Indian Institute of Technology Delhi); Manik Varma (Microsoft Research India, Indian Institute of Technology Delhi); Anish Arora (The Ohio State University); ACM BuildSys 2019

• Smart Surface Classification for Accessible Routing through Built Environment - A Crowd-sourced Approach; Md Osman Gani, Vaskar Raychoudhury (Miami University, Oxford, OH); Janick Edinger (University of Mannheim); Valeria Mokrenko, Zheng Cao, Ce Zhang (Miami University, Oxford, OH); ACM BuildSys 2019

• COLTRANE: ConvolutiOnaL TRAjectory NEtwork for DeepMap Inference; Arian Prabowo (RMIT University); Piotr Koniusz (Data61); Wei Shao, Flora Dilys Salim (RMIT University); ACM BuildSys 2019

• City Scale Monitoring of On-Street Parking Violations with StreetHAWK; Alok Ranjan (Virginia Commonwealth University); Prasant Misra, Arunchandar Vasan (TCS Research & Innovation); Sunil Krishnakumar (TCS (Research & Innovation); Anand Sivasubramaniam (The Pennsylvania State University); ACM BuildSys 2019 Pretom Roy Ovi

Thermal Comfort

• Hot or Not: Leveraging Mobile Devices for Ubiquitous Temperature Sensing; Joseph Breda, Amee Trivedi, Chulabhaya Wijesundara, Phuthipong Bovornkeeratiroj (University of Massachusetts, Amherst); David Irwin (University of Massachusetts Amherst); Prashant Shenoy, Jay Taneja (University of Massachusetts, Amherst); ACM BuildSys 2019

• Heterogeneous Transfer Learning for Thermal Comfort Modeling; Weizheng Hu, Yong Luo (Nanyang Technological University); Zongqing Lu (Peking University); Yonggang Wen (Nanyang Technological University); ACM BuildSys 2019

• Skin Temperature Extraction Using Facial Landmark Detection and Thermal Imaging for Comfort Assessment; Ashrant Aryal, Burcin Becerik-Gerber (University of Southern California); ACM BuildSys 2019 Antonios Xenakis

• OccuTherm: Occupant Thermal Comfort Inference using Body Shape Information; Jonathan Francis (Carnegie Mellon University, Bosch Research and Technology Center); Matias Quintana (National University of Singapore); Nadine von Frankenberg (Technical University of Munich); Sirajum Munir (Bosch Research and Technology Center); Mario Berges (Carnegie Mellon University); ACM BuildSys 2019

• mORAL: An mHealth model for inferring Oral Hygiene Behaviors in-the-wild using wrist-worn inertial sensors Saydeh Karabatis

• FastDeepIoT: Towards Understanding and Optimizing Neural Network Execution Time on Mobile and Embedded Devices, ACM Sensys'18 Emon Dey

• Wind Turbine Surface Damage Detection by Deep Learning Aided Drone Inspection Analysis, MPDI Energies, February 2019 Masud Ahmed

Sample Research Project Reports:

David Welsh and Nirmalya Roy. Smartphone-based Mobile Gunshot Detection, In Proceedings of the 13th Workshop on Context and Activity Modeling and Recognition (CoMoRea’17), co-located with PerCom, March 2017. [pdf]

H M Sajjad Hossain, Md Abdullah Al Hafiz Khan, and Nirmalya Roy. SoccerMate: A Personal Soccer Attribute Profiler using Wearables, in Proceedings of the 1st IEEE PerCom International Workshop on Behavioral Implications of Contextual Analytics (BICA), co-located with PerCom, March 2017. [pdf]

Abu Zaher Md Faridee, Sreenivasan Ramasamy Ramamurthy, H M Sajjad Hossain, and Nirmalya Roy. HappyFeet: Recognizing and Assessing Dance on the Floor, in Proceedings of the ACM 19th International Workshop on Mobile Computing Systems and Applications (HotMobile), Feb. 2018 [pdf]

Varun Mandalapu, Lavanya Elluri and Nirmalya Roy. Developing Machine Learning based Predictive Models for Smart Policing, in Proceedings of the 1st IEEE International Students Workshop on Smart Computing (SmartStudents), co-located with SmartComp, pp. 1-6, Washington D.C., June 2019