Radar-Based Human Skeleton Estimation with CNN-LSTM Network Trained with Limited Data

Mohammad Mahbubur Rahman; Dario Martelli; Sevgi Z Gurbuz

Abstract

Radar-based human activity recognition has opened up new opportunities in the design of cyber-physical systems (CPS) for health and safety by providing an ambient, non-contact, non-intrusive way to monitor human movement at any time of the day (24/7). This is important because it can enable the development of RF-based techniques for the early diagnosis and post-treatment monitoring of ailments resulting in symptoms impacting gait, as well as in improving ageing- in-place and quality life by providing gait-based assessments of fall risk - all in a home environment, where the person monitored would be moving in a natural fashion while doing daily activities. As such, it can provide a more realistic assessment of human mobility and gait, where quantitative gait analysis (QGA) methods are often unavailable and a person may not necessarily be walking in this same way as one does outside a doctor's office. Moreover, RF technologies have have the potential to improve the accessibility of care while also reducing healthcare costs. This paper presents a novel framework for human pose estimation using millimeter-wave (mmWave) radar technology, focusing on personalized healthcare applications. The proposed framework utilizes range-azimuth, range-elevation, and range-Doppler maps as inputs to a convolutional neural network (CNN) with a long short-term memory (LSTM) architecture to capture temporal dependencies and achieve improved skeleton estimation. Furthermore, this paper addresses the limitations of current radar-based skeleton estimation techniques, such as inconsistent kinematics and reliance on sparse radar point clouds. Skeleton estimation accuracy attained using diversified simulations is compared with that achieved real RF data, as validated using gold standard Vicon motion capture (MoCap) measurements as ground truth. The results highlight the potential of mmWave radar-based human skeleton estimation for advancing personalized healthcare and improving gait analysis and fall risk assessment.

Keywords: radar; skeleton estimation

Links
[Full text PDF][Bibtex]