<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "https://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" article-type="research-article" dtd-version="1.2" xml:lang="en">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">IJMABHR</journal-id>
<journal-title-group><journal-title>International Journal of Medical and All Body Health Research</journal-title><abbrev-journal-title>IJMABHR</abbrev-journal-title></journal-title-group>
<issn pub-type="epub">2582-8940</issn>
<publisher><publisher-name>Anfo Publication House</publisher-name></publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="publisher-id">278</article-id>
<article-id pub-id-type="doi">10.54660/IJMBHR.2025.6.1.01-11</article-id>
<title-group><article-title>Deep Learning-Based Real-Time Data Quality Assessment and Anomaly Detection for Large-Scale Distributed Data Streams</article-title></title-group>
<contrib-group>
<contrib contrib-type="author"><name><surname>Zhang</surname><given-names>Hanqing</given-names></name></contrib>
<contrib contrib-type="author"><name><surname>Jia</surname><given-names>Xuzhong</given-names></name></contrib>
<contrib contrib-type="author"><name><surname>Chen</surname><given-names>Chen</given-names></name></contrib>
</contrib-group>
<pub-date publication-format="electronic" date-type="pub"><month>01</month><year>2025</year></pub-date>
<volume>6</volume>
<issue>1</issue>
<fpage>01</fpage>
<lpage>11</lpage>
<permissions><license license-type="open-access" xlink:href="https://creativecommons.org/licenses/by-nc-sa/4.0/?utm"><license-p>CC BY-NC-SA 4.0</license-p></license></permissions>
<self-uri xlink:href="https://www.allmedicaljournal.com/article/278/deep-learning-based-real-time-data-quality-assessment-and-anomaly-detection-for-large-scale"/>
<self-uri content-type="pdf" xlink:href="https://www.allmedicaljournal.com/article/278/download"/>
<abstract><p>Time delay and data quality degradation pose significant challenges in large-scale distributed data streams processing. This paper proposes a deep learning-based real-time data quality assessment and anomaly detection method for distributed streaming data environments. The proposed approach integrates quality-aware feature extraction with adaptive deep neural networks to enable real-time quality monitoring and anomaly detection. A multi-dimensional quality assessment framework is developed, incorporating temporal-spatial correlations and stream characteristics for comprehensive quality evaluation. The system implements a distributed architecture with parallel processing capabilities, enabling scalable operations across multiple nodes while maintaining low-latency responses. A novel online learning mechanism is introduced to adapt model parameters dynamically, ensuring robust performance under evolving data patterns. Experimental evaluation conducted on three large-scale datasets, including industrial IoT sensors (2.5TB), network traffic (1.8TB), and financial transactions (3.2TB), demonstrates superior performance compared to traditional methods. The system achieves 97.8% detection accuracy while maintaining processing latency below 10ms, with linear scalability up to 128 nodes. Results show consistent performance improvement across different operational scenarios, with 95% precision in anomaly detection and throughput exceeding 1.2 million events per second.</p></abstract>
</article-meta>
</front>
</article>