Classification of Streaming Time Series Based on Neural Network Technologies and Behavioral Patterns
DOI:
https://doi.org/10.14529/cmse240305Keywords:
time series, time series classification, autoencoder, time series behavioral patterns (snippets), neural networksAbstract
Data mining of streaming time series is a topical task that occurs in a wide range of subject areas. This article presents the SALTO method (Snippet and Autoencoder based Labeling of Time series coming Online), which combines a neural network model for classifying streaming time series and an analytical method for automatic labeling of training set based on behavioral patterns (snippets). The lightweight architecture of the neural network model used makes it possible to achieve low latency when classifying streaming data. The method involves two stages: preprocessing and classification. At the preprocessing stage, the Preprocessor searches for the optimal value of the length of the subsequence of the input series and performs its labeling. The resulting labels are used to create a training set for the Extractor. The Extractor performs the extraction of the snippet of the input subsequence and assigns it a class based on the similarity of the extracted snippet with snippets from the training set. The experimental results showed that the proposed method performs the classification of a single subsequence in less than 10 ms, which allows SALTO, unlike other competitors, to be used in industrial Internet of Things applications with high performance requirements in accordance with the URLLC (Ultra-Reliable Low Latency Communications) standard.References
Kumar S., Tiwari P., Zymbler M. Internet of Things is a revolutionary approach for future technology enhancement: a review. Journal of Big Data. 2019. Dec. Vol. 6, no. 1. DOI: 10.1186/s40537-019-0268-2.
Ali Nemer M., Azar J., Demerjian J., et al. A Review of Research on Industrial Time Series Classification for Machinery based on Deep Learning. 2022 4th IEEE Middle East and North Africa COMMunications Conference (MENACOMM). IEEE, Dec. 2022. P. 89–94. DOI: 10.1109/menacomm57252.2022.9998277.
Gratius N., Wang Z., Hwang M.Y., et al. Digital Twin Technologies for Autonomous Environmental Control and Life Support Systems. Journal of Aerospace Information Systems. 2024. Apr. Vol. 21, no. 4. P. 332–347. DOI: 10.2514/1.i011320.
Kraeva Ya.A. Anomaly Detection in Digital Industry Sensor Data Using Parallel Computing. Bulletin of the South Ural State University. Series: Computational Mathematics and Software Engineering. 2023. Vol. 12, no. 2. P. 47–61. DOI: 10.14529/cmse230202.
Serrà J., Pascual S., Karatzoglou A. TS 123 501 - V15.2.0 - 5G; System Architecture for The 5G System (5GS) (3GPP TS 23.501 Version 15.2.0 Release 15). 2018.
Wang Z., Yan W., Oates T. Time series classification from scratch with deep neural networks: A strong baseline. 2017 International Joint Conference on Neural Networks (IJCNN). IEEE, May 2017. DOI: 10.1109/ijcnn.2017.7966039.
Dempster A., Petitjean F., Webb G.I. ROCKET: Exceptionally Fast and Accurate Time Series Classification Using Random Convolutional Kernels. Data Mining and Knowledge Discovery. 2020. Vol. 34, no. 5. P. 1454–1495. DOI: 10.1007/s10618020-00701-z.
He K., Zhang X., Ren S., Sun J. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, June 2016. DOI: 10.1109/cvpr.2016.90.
Ismail Fawaz H., Lucas B., Forestier G., et al. InceptionTime: Finding AlexNet for time series classification. Data Mining and Knowledge Discovery. 2020. Sept. Vol. 34, no. 6. P. 1936–1962. DOI: 10.1007/s10618-020-00710-y.
Zymbler M.L., Goglachev A.I. PaSTiLa: Scalable Parallel Algorithm for Unsupervised Labeling of Long Time Series. Lobachevskii Journal of Mathematics. 2024. Mar. Vol. 45, no. 3. P. 1333–1347. DOI: 10.1134/s1995080224600766.
Imani S., Madrid F., Ding W., et al. Introducing time series snippets: a new primitive for summarizing long time series. Data Mining and Knowledge Discovery. 2020. July. Vol. 34, no. 6. P. 1713–1743. DOI: 10.1007/s10618-020-00702-y.
Faouzi J. Time Series Classification: A review of Algorithms and Implementations. Machine Learning (Emerging Trends and Applications) / ed. by K. Kotecha. Proud Pen, 2022. URL: https://inria.hal.science/hal-03558165.
Ye L., Keogh E. Time series shapelets: a new primitive for data mining. Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, June 2009. P. 947–956. KDD09. DOI: 10.1145/1557019.1557122.
Marwan N., Carmenromano M., Thiel M., Kurths J. Recurrence plots for the analysis of complex systems. Physics Reports. 2007. Jan. Vol. 438, 5–6. P. 237–329. DOI: 10.1016/j.physrep.2006.11.001.
Schäfer P., Leser U. Fast and Accurate Time Series Classification with WEASEL. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. ACM, Nov. 2017. CIKM ’17. DOI: 10.1145/3132847.3132980.
Ismail Fawaz H., Forestier G., Weber J., et al. Deep learning for time series classification: a review. Data Mining and Knowledge Discovery. 2019. Mar. Vol. 33, no. 4. P. 917–963. DOI: 10.1007/s10618-019-00619-1.
Serrà J., Pascual S., Karatzoglou A. Towards a universal neural network encoder for time series. 2018. DOI: 10.48550/ARXIV.1805.03908.
Hüsken M., Stagge P. Recurrent neural networks for time series classification. Neurocomputing. 2003. Jan. Vol. 50. P. 223–235. DOI: 10.1016/s0925-2312(01)00706-8.
Hochreiter S. The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems. 1998. Apr. Vol. 6, no. 2. P. 107–116. DOI: 10.1142/s0218488598000094.
Imani S., Madrid F., Ding W., et al. Matrix Profile XIII: Time Series Snippets: A New Primitive for Time Series Data Mining. 2018 IEEE International Conference on Big Knowledge, ICBK 2018, Singapore, November 17-18, 2018 / ed. by X.Wu, Y. Ong, C.C. Aggarwal, H. Chen. IEEE Computer Society, 2018. P. 382–389. DOI: 10.1109/ICBK.2018.00058.
Gharghabi S., Imani S., Bagnall A.J., et al. An ultra-fast time series distance measure to allow data mining in more complex real-world deployments. Data Min. Knowl. Discov. 2020. Vol. 34, no. 4. P. 1104–1135. DOI: 10.1007/s10618-020-00695-8.
Li P., Pei Y., Li J. A comprehensive survey on design and application of autoencoder in deep learning. Applied Soft Computing. 2023. May. Vol. 138. P. 110176. DOI: 10.1016/j.asoc.2023.110176.
Zymbler M., Goglachev A. Fast Summarization of Long Time Series with Graphics Processor. Mathematics. 2022. May. Vol. 10, no. 10. P. 1781. DOI: 10.3390/math10101781.
Chung J., Gulcehre C., Cho K., Bengio Y. Gated Feedback Recurrent Neural Networks. Proceedings of the 32nd International Conference on Machine Learning. Vol. 37 / ed. by F. Bach, D. Blei. Lille, France: PMLR, July 2015. P. 2067–2075. Proceedings of Machine Learning Research. URL: https://proceedings.mlr.press/v37/chung15.html.
Dumoulin V., Visin F. A guide to convolution arithmetic for deep learning. 2016. DOI: 10.48550/ARXIV.1603.07285.
Ermshaus A., Schäfer P., Leser U. ClaSP: parameter-free time series segmentation. Data Mining and Knowledge Discovery. 2023. Vol. 37. P. 1262–1300. DOI: 10.1007/s10618-023-00923-x.
Jorge Reyes-Ortiz D.A. Human Activity Recognition Using Smartphones. 2013. DOI: 10.24432/C54S4K.
Bilenko R., Dolganina N., Ivanova E., Rekachinsky A. High-performance Computing Resources of South Ural State University. Bulletin of the South Ural State University. Computational Mathematics and Software Engineering. 2022. Vol. 11, no. 1. P. 15–30. (in Russian) DOI: 10.14529/cmse220102.
Minor B.D., Doppa J.R., Cook D.J. Learning Activity Predictors from Sensor Data: Algorithms, Evaluation, and Applications. IEEE Transactions on Knowledge and Data Engineering. 2017. Dec. Vol. 29, no. 12. P. 2744–2757. DOI: 10.1109/tkde.2017.2750669.
Lubba C.H., Sethi S.S., Knaute P., et al. catch22: CAnonical Time-series CHaracteristics: Selected through highly comparative time-series analysis. Data Mining and Knowledge Discovery. 2019. Aug. Vol. 33, no. 6. P. 1821–1852. DOI: 10.1007/s10618-019-00647-x.
