https://doi.org/10.1140/epjds/s13688-024-00512-y
Research
Robustness of topological persistence in knowledge distillation for wearable sensor data
1
Department of Computer Science and Engineering, Seoul National University of Science and Technology, 01811, Seoul, Republic of Korea
2
Lawrence Livermore National Laboratory, 94550, Livermore, CA, USA
3
Department of Computer Science and Engineering, University of Nevada, 89557, Reno, NV, USA
4
Department of Epidemiology and Biostatistics, University of South Carolina, 29208, Columbia, SC, USA
5
College of Health Solutions, Arizona State University, 85004, Phoenix, AZ, USA
6
School for Engineering of Matter, Transport and Energy, 85281, Tempe, AZ, USA
7
Geometric Media Lab, School of Arts, Media and Engineering and School of Electrical, Computer and Energy Engineering, Arizona State University, 85281, Tempe, AZ, USA
Received:
13
July
2023
Accepted:
22
November
2024
Published online:
20
December
2024
Topological data analysis (TDA) has shown great success in various applications involving wearable sensor data. However, there are difficulties in leveraging topological features in machine learning and wearable sensors because of the large time consumption and computational resources required to extract the features. To address this problem, knowledge distillation (KD) is utilized to generate a small model and accommodate topological features with persistence image (PI) representations from the raw time series data. Deploying topological knowledge in KD enables the student to achieve better performance compared to the one trained solely on raw time series data. However, it is not yet known if there are coherent characteristics for topological features in PI, which can aid in improving the performance during KD. In this paper, we investigate the suitability and challenges of utilizing topological features in KD for wearable sensor data, thereby contributing to the advancement of the field. Our study explores the impact of transferred topological features by comparing the Teacher-to-Student framework with Multiple Teachers-to-Student where teachers utilize both time series data and persistence images obtained by TDA as inputs. Additionally, we conduct a rigorous examination of topological knowledge effects by testing under various corruptions, knowledge types, and learning strategies in the context of human activity recognition tasks. Our analysis of topological features in KD presents the optimal strategy for incorporating these features. This study includes datasets of varying scales, window lengths, and activity classes, providing a comprehensive evaluation. Our results demonstrate that leveraging topological features in KD to enhance performance across databases.
Key words: Knowledge distillation / Topological data analysis / Persistence image / Wearable sensor data
© The Author(s) 2024
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
