IS 698/800: Smart Home Health Analytics — Fall 2016

Times: Monday 4:30pm – 7:00pm
Location: Performing Arts & Humanities 108
Instructor: Nirmalya Roy
Instructor’s Office Location and Hours: ITE 421 Monday 1:00 – 2:30pm, or by appointment
Instructor’s Email: nroy at umbc dot edu

Course Webpage:

Course Descriptions: This course will examine the machine learning techniques, different systems and toolsets that are needed to recognize, discover, and learn the human activities and behaviors and their impact on human health and wellness in smart home environment. Human activity recognition is a growing field of research with its far reaching impact on proactive health care and physical fitness. We will discuss different machine learning algorithms, evaluation metrics, and their application to human activity and behavior recognition. A special emphasize will be given to the techniques with respect to a variety of emerging smart environment contexts spanning across monitoring activities of daily living (ADLs), physiological signals (heart rate, galvanic skin response) and psychological behaviors (stress, depression, agitation) etc. These human signals can be measured using wearable and IoT (Internet-of-Things) devices such as smart phones and wristwatches to keep people informed about their activity, life-style, health, mood, behavior, and surrounding. Students are required to do a group project. The tools necessary to develop the project will be made available and reviewed in class, but students are expected to be comfortable with basic use of software, smart phones and smart wristwatches and have an interest in using technology for smart home health assessment.

Course Objectives: The purpose of this course is to provide a comprehensive foundation to apply machine learning, data science, and statistical learning methodologies in solving problems in real life applications such as smart home healthcare. This course should enhance students’ reasoning, problem-solving and modeling abilities, particularly in dealing with data science problems. More specifically, the course has the following objectives:

  • Familiarize students with the concepts and applications of machine and statistical learning techniques to solve real life data science problems.
  • Teach students how to think and formalize smart environments research problems with real data and what computational techniques to apply.

We will not assume any background beyond high school level mathematics and familiarity with programming concepts. However, students are expected to spend time in learning the concepts in this course, many of which will be covered in details.

Course Topics:

  • Gerontechnology
  • Supervised Learning
  • Bayesian Decision Theory
  • Parametric Methods
  • Decision Theory
  • Machine Learning Evaluation
  • Machine Learning Toolkits
  • Activity Learning
  • Smart Home Health Technologies
  • Functional and Behavioral Health Assessment
  • Qualitative and Quantitative Clinical Health Assessment Tools
  • Applications

Course Prerequisites: IS 733 (Data Mining) or consent of the instructor

Instructional Materials: Given that this is an interdisciplinary IS course cross cutting machine learning, data science, Internet of Things, smart home technologies and clinical psychology, there are no single textbook that fully cover the integrated aspects of the course material. At the first half of the course the instructor will discuss the core methodologies behind the course and in the second half the students will be reading various research articles to assimilate the emerging trend in the subject matter of smart home health analytics. A list of research articles will be made available at the course webpage. There are a few books that are relevant to this course are listed next.

Recommended Textbooks (Optional):

Course Project: Students are required to do a group project (two students in each group) as part of this course work. Each team is required to propose a study hypothesis, collect their own datasets over a period of time using smartphones and smartwatches for a specific smart environments application as approved by the instructor and use of appropriate data analytics and machine learning methodology to recognize and evaluate the context of the users.

Reading and Writing Assignment: Each student will be required to present 1 research paper in the entire semester and write 3 critiques of the 3 distinct research papers as being presented by the other students in the class.

Course Requirements and Grading:

Course Participation 5%
Research Paper Presentation 5%
Critique Writing 5%
Homework  20%
1 Midterm Exam  25%
Research Project & Final Report (RR: 5% …..)  40%

Tentative Course Schedule:
(Subject to change as the semester progresses)

Week Date Topic Handout/Assignment Due Notes
 1  9/12 Course Introduction, Review of Probability & Statistics Course Syllabus Introduction 

Probability Review

 2  9/19 Machine Learning: Introduction

Weka Tutorial

Homework 1 Introduction ML (Textbook IML: Chapter 1, Sections  1.1 and  1.2)

Weka Tutorial

Example Datasets

Research Reflection Sign Up Sheet

 3  9/26 Supervised Learning Homework 1 Supervised (Textbook IML: Chapter 2, Sections  2.1,  2.3, 2.4, 2.5, 2.7, and 2.8)

Research Logistics

Research Reflection

 4  10/3 Bayesian Decision Theory, Research Reflection Discussion Presentation & Critique Logistics

Homework 2

Bayesian (Textbook IML: Chapter 3, Sections  3.1,  3.2 and 3.6)


 5  10/10 Parametric Methods, Research Reflection Discussion Parametric
 6  10/17 Research Reflection Discussion Homework 3 Homework 2
 7  10/24 Decision Trees,

Research Paper Presentation

Decision Trees

Arpita Roy, Anamika Paul RupaRiyaz Habibi,

 8  10/31 Research Paper Presentation Sensor Data Collection Tutorial Homework 3 Onimi Jademi,    Yueyang Jiang, Sajjad HossainRonak Razavisousan,  Naveena Sureshkumar,
9 11/7 Research Paper Presentation Bipen BasnyatHafiz Khan,  Ankit Panwar,     Nirajkumar Lohbare, Sarthak Bhatt, Sarthak Pathak
 10  11/14  

Research Paper Presentation


Sonali Singh,   Rasika SalviParikshitha  Manjunath,     Olumuyiwa SomideBharath Jegannathan,   Rishita Amin,   Prasad Chaudhari,   Krupa Adhia
11 11/21 Evaluation

Exam Review

Homework 4 Evaluation

Exam Review

12 11/28 Exam
13 12/5 Final Research Project Presentation Homework 4 Sajjad (Team 1)
Krupa + Parikshita (Team 5)
Sarthak + Sarthak (Team 8)
Prasad + Niraj (Team 12)
Riyaz + Ankit (Team 7)
Ronak + Sonali (Team 6)
14 12/12 Final Research Project Presentation Final Project Report Template

Final Project Report Logistic
Sample IEEE Style File

Arpita + Anamika (Team 2)
Bipen + Rasika  (Team 3)
Rishita + Naveena  (Team 9)
Yueyang (Team 11)
Hafiz (Team 4)
Olumuyiwa + Bharath + Onimi (Team 10)

Course Resources:

Smartphone and Smartwatch Data Collection Apps:

iPhone Users:

Android Users:

Research Paper Presentation Schedule:
(To be determined)

Group No. Team Members Project Title Devices
1 Sajjad Hossain Soccer Mate: A personal soccer attribute profiling using wrist worn devices Actigraph, Zepp
2 Anamika Rupa, Arpita Roy Are you a foodie? Detecting the spiciness of your food using smart watch Empatica
3 Bipen Basnyat, Rasika Salvi Assessing effectiveness of  machine learning techniques in Personal Weight & Diet Management using Android Based Apps FLIR One, Android phone
4 Hafiz Khan Does smoking reduce stress? Stress analysis through the physiological signal of a smoker. Microsoft Band, Empatica
5 Krupa Adhia, Parikshitha Manjunath Watch Your Weight Hapifork, Smart Scale
6 Ronak Razavisousan, Sonali Singh The effect of phone activity during the bed time in sleep quality iPhone, Sleep App
7 Riyaz Habibi, Ankit Panwar Comparative study of caloric expenditure between wearable devices and smartphones Fitbit, Microsoft Band
8 Sarthak Bhatt, Sarthak Pathak Music based interactive detection and treatment for stress Microsoft Band
9 Rishita Amin, Naveena Sureshkumar Smartwatch Based Diet Monitoring Samsung Gear
10 Olumuyiwa Somide, Bharath Jegannathan, Onimi Jademi Analyzing ambient and body thermal detection technologies to regulate the optimal room temperature during sleep, using common smart devices and wearable sensors Apple Watch, Microsoft Band, iPhone
11 Yueyang  Jiang A Context-aware Medication Reminder System for Adult Day Care Center Smart wristband (K18)
12 Prasad Chaudhari, Nirajkumar Lobhare Identification of health condition on the basis of respiratory syndrome detection Microsoft Band

Relevant conferences and workshops in the broad area of  the course material (machine learning, data mining, pervasive and ubiquitous computing, pervasive health, health IT, HCC):

  • IEEE ICDM, ICDE, PerCom, ICML, Wireless Health, EMBC
  • ACM KDD, SenSys, IPSN, Ubicomp, MobiSys, CHI
  • AAAI, Pervasive Health, ICOST, IE, WristSense

Reading List:

Analysis of Long-term Abnormal Behaviors for Early Detection of Cognitive Decline,
Daniele Riboni, Gabriele Civitarese, Claudio Bettini, IEEE PASTA 2016

Elderly Medication Adherence Monitoring with the Internet of Things,
Xiaoping Toh, Hwee-Xian Tan, Huiguang Liang, Hwee Pink Tan, IEEE PASTA 2016

Beacon-Based Multi-Person Activity Monitoring System for Day Care Center   (Yueyang Jiang)
Kiyoaki Komai, Manato Fujimoto, Yukata Arakawa, Suwa Hirohiko, Yukitoshi Kashimoto,
Keiichi Yasumoto, IEEE PASTA 2016

Improving the Sensitivity of Unobstrusive Inactivity Detection in Sensor-Enabled Homes for the Elderly
Alvin C Valera, Hwee Pink Tan, Liming Bai, IEEE PASTA 2016

On the impact of activity recognition in monitoring cognitive decline
Claudio Bettini, Keynote Talk, IEEE PerCom 2016 workshop

Smart Devices are Different: Assessing and Mitigating Mobile Sensing Heterogeneities for Activity Recognition    (Olumuyiwa Somide)
Allan Stisen, Henrik Blunck, Sourav Bhattacharya, Thor Siiger Prentow, Mikkel Baun Kjaergaard, Anind Dey, Tobias Sonne, and Mads Moller Jensen, ACM SenSys 2015

Connecting Personal-scale Sensing and Networked Community Behavior to Infer Human Activities
Nicholas D. Lane, Li Pengyu, Lin Zhou, Feng Zhao, ACM Ubicomp 2014

SymDetector: detecting sound-related respiratory symptoms using smartphones   (Nirajkumar Lohbare)
Xiao Sun, Zongqing Lu, Wenjie Hu, Guohong Cao, ACM Ubicomp 2015

Evaluating tooth brushing performance with smartphone sound data   (Prasad Chaudhari)
Joseph Korpela, Ryosuke Miyaji, Takuya Maekawa, Kazunori Nozaki, Hiroo Tamagawa; ACM Ubicomp 2015

SleepTight: low-burden, self-monitoring technology for capturing and reflecting on sleep behaviors
Eun Kyoung Choe, Bongshin Lee, Matthew Kay, Wanda Pratt, Julie A. Kientz, ACM Ubicomp 2015

RunBuddy: a smartphone system for running rhythm monitoring   (Ankit Panwar)
Tian Hao, Guoliang Xing, Gang Zhou; ACM Ubicomp 2015

MyoVibe: vibration based wearable muscle activation detection in high mobility exercises
Frank Mokaya, Roland Lucas, Hae Young Noh, Pei Zhang, ACM Ubicomp 2015

DoppleSleep: a contactless unobtrusive sleep sensing system using short-range Doppler radar   (Sonali Singh)
Tauhidur Rahman, Alexander T. Adams, Ruth Vinisha Ravichandran, Mi Zhang, Shwetak N. Patel, Julie A. Kientz, Tanzeem Choudhury; ACM Ubicomp 2015

DeepEar: robust smartphone audio sensing in unconstrained acoustic environments using deep learning   (Rasika Salvi)
Nicholas D. Lane, Petko Georgiev, Lorena Qendro, ACM Ubicomp 2015

puffMarker: a multi-sensor approach for pinpointing the timing of first lapse in smoking cessation,
Nazir Saleheen, Amin Ahsan Ali, Syed Monowar Hossain, Hillol Sarker, Soujanya Chatterjee, Benjamin Marlin, Emre Ertin, Mustafa al’Absi, Santosh Kumar; ACM Ubicomp 2015

I did not smoke 100 cigarettes today!: avoiding false positives in real-world activity recognition,
Le T. Nguyen, Ming Zeng, Patrick Tague, Joy Zhang; ACM Ubicomp 2015

Trajectories of depression: unobtrusive monitoring of depressive states by means of smartphone mobility traces analysis,
Luca Canzian, Mirco Musolesi; ACM Ubicomp 2015

Invisible Sensing of Vehicle Steering with Smartphones
Dongyao Chen, Kyong-Tak Cho, Sihui Han, Zhizhuo Jin, Kang G. Shin, ACM MobiSys 2015

Recognizing Social Gestures with a Wrist-Worn SmartBand
Jamie Payton, Jonathan S Knighten, Stephen McMillan and Tori Chambers, IEEE WristSense 2015

Gesture Control By Wrist Surface Electromyography
Abhishek Nagar and Xu Zhu, IEEE WristSense 2015

Gait, Wrist and Sensors: Detecting Freezing of Gait in Parkinson’s Disease from Wrist Movement
Sinziana Mazilu, Ulf Blanke, Gerhard Tröster, IEEE WristSense 2015

The Case for Smartwatch-based Diet Monitoring    (Rishita Amin)
Sougata Sen, Vigneshwaran Subbaraju, Archan Misra, Rajesh K Balan and Youngki Lee, IEEE WristSense 2015

Towards Detection of Bad Habits by Fusing Smartphone and Smartwatch Sensors
Muhammad Shoaib, Stephan Bosch, Ozlem Durmaz Incel, Hans Scholten and Paul Havinga, IEEE WristSense 2015

Detecting Self-harming Activities with Wearable Devices
Levi Malott, Sriram Chellappan, Pratool Bharti, Nicholas Hilbert, Ganesh Gopalakrishna, IEEE WristSense 2015

Never Skip Leg Day: A Novel Wearable Approach to Monitoring Gym Leg Exercises
Bo Zhou, Mathias Sundholm, Jingyuan Cheng and Heber Cruz, Paul Lukowicz, IEEE PerCom 2016

IRIS: Tapping Wearable Sensing to Capture In-Store Retail Insights on Shoppers
Meeralakshmi Radhakrishnan, Sharanya Eswaran, Archan Misra, Deepthi Chander, Koustuv Dasgupta, IEEE PerCom 2016

MT-Diet: Automated Smartphone based Diet Assessment with Infrared Images    (Bipen Basnyat)
Junghyo Lee, Ayan Banerjee and Sandeep Gupta, IEEE PerCom 2016

A Probabilistic model for early prediction of abnormal clinical events using vital sign correlations in home-based monitoring
Abdur Rahim Mohammad Forkan, Ibrahim Khalil, IEEE PerCom 2016

HealthyOffice: Mood Recognition At Work Using Smartphones and Wearable Sensors  (Anamika Paul Rupa)
Alexandros Zenonos, Aftab Khan, Georgios Kalogridis, Stefanos Vatsikas, Tim Lewis, Mahesh Sooriyabandara, IEEE WristSense 2016

From Smart to Deep: Robust Activity Recognition on Smartwatches Using Deep Learning  (Arpita Roy)
Sourav Bhattacharya, Nicholas D. Lane, IEEE WristSense 2016

Did You Take a Break Today?: Detecting Playing Foosball Using Your Smartwatch
Sougata Sen, Kiran K. Rachuri, Abhishek Mukherji, Archan Misra, IEEE WristSense 2016

weSport: Utilising Wrist-Band Sensing to Detect Player Activities in Basketball Games
Lu Bai, Christos Efstratiou, Chee Siang Ang, IEEE WristSense 2016

WatchUDrive: Differentiating Drivers and Passengers Using Smartwatches
Alex Mariakakis, Vijay Srinivasan, Kiran Rachuri, Abhishek Mukherji, IEEE WristSense 2016

Recognizing Text Using Motion Data From a Smartwatch
Luca Arduser, Pascal Bissig, Philipp Brandes, Roger Wattenhofer, IEEE WristSense 2016

Automated detection of activity transitions for prompting
Feuz K., Robertson K., Rosasco C., Cook D. & Schmitter-Edgecombe, M. IEEE Transactions on Human-Machine Systems, 2015

Mobile Phone Sensor Correlates of Depressive Symptom Severity in Daily-Life Behavior: An Exploratory Study  (Onimi Jademi)
Saeb S, Zhang M, Karr CJ, Schueller SM, Corden ME, Kording KP, Mohr DC, J Med Internet Res 2015

Re-Active Learning: Active Learning With Relabeling     (Sajjad Hossain)
Christopher H. Lin, Mausam, Daniel S. Weld, AAAI 2016

iSleep: unobtrusive sleep quality monitoring using smartphones   (Ronak Razavisousan)
Tian Hao, Guoliang Xing, Gang Zhou, ACM SenSys ’13

EmotionCheck: Leveraging Bodily Signals and False Feedback to Regulate our Emotions   (Naveena Sureshkumar)
J. Costa, A. T. Adams, M. F. Jung, F. Guimbetiere, and T. Choudhury, ACM UbiComp 2016

Convolutional Neural Networks for Human Activity Recognition using Mobile Sensors   (Hafiz Khan)
Ming Zeng, Le T. Nguyen, Bo Yu, Ole J. Mengshoel, Jiang Zhu, Pang Wu, Joy Zhang; ACM MobiCASE 2014

Accurate Caloric Expenditure of Bicyclists using Cellphones    (Riyaz Habibi)
Andong Zhan, Marcus Chang, Yin Chen, Andreas Terzis, ACM SenSys 2012

A new method for measuring meal intake in humans via automated wrist motion tracking  (Krupa Adhia)
Y. Dong, A. Hoover, J. Scisco, and E. Muth, Applied psychophysiology and biofeedback, 2012

Analysis of chewing sounds for dietary monitoring   (Parikshitha Manjunath)
O. Amft, M. Stager, P. Lukowicz, and G. Troster, UbiComp’05

Smart Phones and Dietary Tracking: A Feasibility Study   (Rasika Salvi)
Barbara Cunningham, MS Thesis, Arizona State University, 2011

Using Heart Rate Monitors to Detect Mental Stress  (Sarthak Bhatt)
Jongyoon Choi and Ricardo Gutierrez-Osuna, ACM Body Sensor Networks, 2009

Wearable Biomedical Measurement Systems for Assessment of Mental Stress of Combatants in Real Time
Fernando Seoane, Inmaculada Mohino-Herranz, Javier Ferreira, Lorena Alvarez, Ruben Buendia, David Ayllón, Cosme Llerena, & Roberto Gil-Pita, IEEE Sensor Journal, 2014

Effects of thermal environment on sleep and circadian rhythm  (Bharath Jegannathan)
Kazue Okamoto-Mizuno and Koh Mizuno, Journal of Physiological Anthropology, 2013

StressSense: Detecting Stress in Unconstrained Acoustic Environments using Smartphones   (Sarthak Pathak)
H. Lu, M, Rabbi, G. Chittaranjan, D. Frauendorfer, M. Mast, A. Campbell, D. Gatica-Perez, T. Choudhury, ACM Ubicomp 2012

Just In Case: Diminished mental health can interfere with optimal academic performance. The source of symptoms might be related to your course work; if so, please speak with me. However, problems with other parts of your life can also contribute to decreased academic performance. UMBC provides cost-free and confidential mental health services through the Counseling Center to help you manage personal challenges that threaten your personal or academic well-being.

Remember, getting help is a smart and courageous thing to do — for yourself and for those who care about you. For more resources get the Just in Case mental health resources Mobile and Web App. This app can be accessed by clicking

The UMBC Counseling Center is in the Student Development & Success Center (between Chesapeake and Susquehanna Halls). Phone: 410-455-2472. Hours: Monday-Friday 8:30am-5:00pm.

Student Support Services: UMBC is committed to eliminating discriminatory obstacles that disadvantage students based on disability. Student Support Services (SSS) is the UMBC department designated to receive and maintain confidential files of disability-related documentation, certify eligibility for services, determine reasonable accommodations, develop with each student plans for the provision of such accommodations, and serve as a liaison between faculty members and students regarding disability-related issues.  If you have a disability and want to request accommodations, contact SSS in the Math/Psych Bldg., room 213 or at 410-455-2459.  SSS will require you to provide appropriate documentation of disability.  If you require accommodations for this class, make an appointment to meet with me to discuss your SSS-approved accommodations.

Academic Integrity: Cheating in any form, will be subject to discipline according to university regulations. Projects that contain plagiarized materials will receive an automatic letter grade of ‘F’. Multiple violations will be handled according to university regulation. Please refer to Academic Integrity for more information.

Acknowledgement: Prof. Diane Cook and Prof. Larry Holder