基于局部与全局信息的自动文摘算法

Research of Automatic Summarization Based on Local and Global Information of Sentences

关键词:  K-means  n-grims  段落聚类  自然语言理解

Key words:  K-means  n-grims  paragraph clustering  natural language understanding

摘要:

采用平均特征词频率策略计算特征词权重,用快速n-grims算法对各特征词所处的概念体进行加权,用一种改进的K-means聚类算法进行段落聚类,提出一种基于局部与全局信息的自动文摘算法并给出算法评估。该算法不仅能够自适应获得k值,而且有效防止了初始点的随机选择对聚类结果的影响。评测结果表明该算法对经济类和科技类文章的准确率和召回率都明显高于新闻类和文学类文章,利用机器文摘进行分类的准确率明显高于使用原文本进行分类。该算法所得到的文摘,在各项指标上都优于传统方法生成的文摘。

Abstract:

The idea of our approach is to exploit both the local and global properties of sentences.In order to obtain local property,we use a term weighting scheme that employs average term frequency in a document as the normalization factor.And a fast algorithm for matching N-grams is uesd to optimize term weighting.The method can obtain an improved K-means method to cluster paragraphs,and discovers thematic areas according to clustering results.Furthermore,it integrates local and global property to produce summarization.And experiments do prove that it is feasible to use the method to develop a domain automatic abstracting system,which is valuable for further study.