Deep Learning-Based Real-Time Data Quality Assessment and Anomaly Detection for Large-Scale Distributed Data Streams
Authors
Article Information
Publication Timeline
- Published
Abstract
Time delay and data quality degradation pose significant challenges in large-scale distributed data streams processing. This paper proposes a deep learning-based real-time data quality assessment and anomaly detection method for distributed streaming data environments. The proposed approach integrates quality-aware feature extraction with adaptive deep neural networks to enable real-time quality monitoring and anomaly detection. A multi-dimensional quality assessment framework is developed, incorporating temporal-spatial correlations and stream characteristics for comprehensive quality evaluation. The system implements a distributed architecture with parallel processing capabilities, enabling scalable operations across multiple nodes while maintaining low-latency responses. A novel online learning mechanism is introduced to adapt model parameters dynamically, ensuring robust performance under evolving data patterns. Experimental evaluation conducted on three large-scale datasets, including industrial IoT sensors (2.5TB), network traffic (1.8TB), and financial transactions (3.2TB), demonstrates superior performance compared to traditional methods. The system achieves 97.8% detection accuracy while maintaining processing latency below 10ms, with linear scalability up to 128 nodes. Results show consistent performance improvement across different operational scenarios, with 95% precision in anomaly detection and throughput exceeding 1.2 million events per second.
How to Cite
Zhang, H., Jia, X., & Chen, C. (2025). Deep Learning-Based Real-Time Data Quality Assessment and Anomaly Detection for Large-Scale Distributed Data Streams. International Journal of Medical and All Body Health Research, 6(1), 01-11. https://doi.org/10.54660/IJMBHR.2025.6.1.01-11
Zhang, Hanqing, et al. "Deep Learning-Based Real-Time Data Quality Assessment and Anomaly Detection for Large-Scale Distributed Data Streams." International Journal of Medical and All Body Health Research, vol. 6, no. 1, 2025, pp. 01-11. https://doi.org/10.54660/IJMBHR.2025.6.1.01-11.
Zhang, H., Jia, X. and Chen, C. (2025) 'Deep Learning-Based Real-Time Data Quality Assessment and Anomaly Detection for Large-Scale Distributed Data Streams', International Journal of Medical and All Body Health Research, 6(1), pp. 01-11. doi: 10.54660/IJMBHR.2025.6.1.01-11.
Zhang H, Jia X, Chen C. Deep Learning-Based Real-Time Data Quality Assessment and Anomaly Detection for Large-Scale Distributed Data Streams. International Journal of Medical and All Body Health Research. 2025;6(1):01-11. doi:10.54660/IJMBHR.2025.6.1.01-11
Similar Articles
- Comparison of Elective Cesarean Section and Vaginal Delivery Outcomes: A Study of 100 Patients at Saraswathi Institute of Medical Sciences Vol 6, Issue 1 (2025)
- Classification, Importance, and Pathogenicity of Facklamia Species and Related Bacteria Vol 6, Issue 1 (2025)
- Knowledge and Attitudes towards Sexually Transmitted Infections among Medical Groups and Non-Medical Colleges at Baghdad University 2024 Vol 6, Issue 1 (2025)
- Pathogenesis and Therapeutic Targets of Cytokine Storms in Bacterial Infections Vol 6, Issue 1 (2025)
- Response Analysis of Teens towards Mentally Ill Patients Vol 6, Issue 1 (2025)
Find this Article on
Google Scholar Semantic Scholar OpenAlex Scilit Crossref ResearchGate BASEA newly published article may take a few days to appear on these services.