基于多质心的不良文本快速过滤方法

A Method of Illegal and Harmful Text Fast Filter Based on Multi-Centroid Vector

关键词:  不良文本  快速过滤  多质心向量  Rocchio  K-means

Key words:  illegal and harmful text  fast filter  multi-centroid vector  Rocchio  K-means

摘要:

针对Rocchio容易受到类别样本分布及噪声影响的而导致错误扩大类别范围的问题,提出对训练样本进行聚类,使用聚类形成的多个簇的质心向量替代单个质心向量作为过滤判定向量组的方法。该方法既能保证过滤效率,又比单质心的Rocchio过滤法具有更高的召回率和准确率。

Abstract:

Aiming at the defect in Rocchio that classification range could be easily mis-extended due to distribution of classification samples and noises,a filtering method is presented in this paper, in which a vector of single centroid is substituted by a vector group of centroids at multiple clusters formed by clustering trained samples and used as a deciding vector group for filtering.This method is characterized by lossless filtering efficiency.Recalling rate and accuracy of this method is higher than that of the single centroid-featured Rocchio Filtering.