基于HTM网络的多源海洋实时观测数据异常检测

Anomaly Detection of Multi-source Marine Real-time Observation Data Based on HTM Network

关键词:  海洋观测系统  数据预处理  HTM网络  数据相关性  异常检测

Key words:  marine observation system  data preprocessing  HTM network  data correlation  anomaly detection

摘要:

在时下海洋观测系统中，往往通过分布于不同地理位置的智能传感器来获取实时观测数据。然而由于通信环境不稳定、观测仪器故障、数据采集或传输软件运行异常等原因，观测数据的完整性、可靠性和时效性往往得不到保障。本文基于层级实时记忆(Hierarchical Temporal Memory，HTM)网络设计多源数据异常检测算法，对不同测点海洋实时观测数据流进行质量监控。首先考虑海洋观测系统中存在的数据缺测、网络丢包现象，对观测数据进行预处理；接着基于HTM网络，生成观测数据的稀疏离散表征，动态更新神经元活跃和预测状态，并根据赫布法则奖励或惩罚突触连通值，模拟时序数据的空间和时间关系，从而学习和识别数据内部特征，实现单源海洋实时数据流的异常检测；最后在此基础上，利用不同测点间观测数据的距离相关性，对多源海洋实时数据流进行质量监控，降低异常数据漏报率。实验结果表明，本文提出的算法能有效检测出海洋实时观测数据异常点，且识别速度快于数据采集速度，能保证异常检测过程的准确性和实时性，符合实际应用需求。

Abstract:

In the current marine observation system,real-time observation data are often obtained by intelligent sensors distributed in different geographical locations.However,due to unstable communication environment,observation instrument failure,abnormal operation of data acquisition or transmission software and other reasons,the integrity,reliability and timeliness of observation data are often not guaranteed.In this article,a multi-source data anomaly detection algorithm based on Hierarchical Temporal Memory (HTM) network is designed to monitor the quality of real-time marine observation data streams at different measuring points.Firstly,considering the phenomenon of data missing and network packet loss in the marine observation system,the observation data is preprocessed.Then,based on the HTM network,the sparse discrete representation of the observed data is generated,and the active and predicted states of the neurons are dynamically updated.According to the Hebb rule,the synaptic connectivity values are rewarded or punished to simulate the spatial and temporal relationship of the time series data,so as to learn and identify the internal characteristics of the data and realize the anomaly detection of the single-source real-time marine data stream.Finally,on this basis,the quality of multi-source marine real-time data stream is monitored to reduce the false negative rate of abnormal data by using the distance correlation of observation data between different measuring points.The experimental results show that the algorithm proposed in this article can effectively detect the abnormal points of marine real-time observation data,and the recognition speed is faster than the data acquisition speed,which can ensure the accuracy and real-time performance of the anomaly detection process and meet the actual application requirements.