Adaptive Real-Time Anomaly Detection and Data Imputation in Multi-Device IoT Streams via Nonlinear Dimensionality Reduction and Dynamic Sliding-Window Statistics
Rahmat Faisal

ESRI Indonesia

Abstract

The rapid growth of diverse IoT deployments has made real-time monitoring of multi-sensor streams essential for ensuring system reliability and data integrity. In this work, we propose a novel framework that combines nonlinear dimensionality reduction with adaptive sliding-window statistical analysis to enable scalable, online anomaly detection and missing data imputation. Sensor data are first compressed via a lightweight autoencoder that captures complex nonlinear correlations across sensors in a compact latent representation. A dynamic sliding-window statistical module then adaptively adjusts the window size and computes key features such as mean, variance, entropy, and reconstruction error from recent latent embeddings. Anomalies are detected using context-sensitive thresholds derived from continuously updated statistical baselines, while missing or corrupted values are imputed using both latent autoencoder outputs and interpolation guided by recent temporal trends.

This architecture is designed for deployment in edge-cloud hybrid environments optimized for resource-constrained IoT devices. The framework supports accurate, low-latency anomaly detection and data repair across heterogeneous IoT streams, adapting to evolving sensor behaviors without relying on predefined thresholds or offline training. This approach provides a robust and scalable solution for maintaining high data quality in complex IoT ecosystems.

Keywords: Anomaly Detection, IoT, Real-Time Anomaly Detection

Topic: Topic C: Emerging Technologies in Remote Sensing