| 引用本文： |
-
刘柏霆,管卫利,李陶深.基于XLNet+BiGRU+Att(Label)的文本分类模型[J].广西科学院学报,2022,38(4):412-419. [点击复制]
- LIU Boting,GUAN Weili,LI Taoshen.Text Classification Model Based on XLNet+BiGRU+Att (Label)[J].Journal of Guangxi Academy of Sciences,2022,38(4):412-419. [点击复制]
|
|
| 摘要: |
| 传统的词向量嵌入模型，如Word2Vec、GloVe等模型无法实现一词多义表达；传统的文本分类模型也未能很好地利用标签词的语义信息。基于此，提出一种基于XLNet+BiGRU+Att(Label)的文本分类模型。首先用XLNet生成文本序列与标签序列的动态词向量表达；然后将文本向量输入到双向门控循环单元(BiGRU)中提取文本特征信息；最后将标签词与注意力机制结合，选出文本的倾向标签词，计算倾向标签词与文本向量的注意力得分，根据注意力得分更新文本向量。通过对比实验，本文模型比传统模型在文本分类任务中的准确率更高。使用XLNet作为词嵌入模型，在注意力计算时结合标签词能够提升模型的分类性能。 |
| 关键词: 文本分类 XLNet BiGRU 标签词 注意力机制 |
| DOI：10.13657/j.cnki.gxkxyxb.20221209.010 |
| 投稿时间：2022-04-15修订日期：2022-07-11 |
| 基金项目:国家自然科学基金项目(61762010)和广西科技计划项目(桂科AD20297125)资助。 |
|
| Text Classification Model Based on XLNet+BiGRU+Att (Label) |
|
LIU Boting1, GUAN Weili1,2, LI Taoshen1,2
|
| (1.School of Computer, Electronics and Information, Guangxi University, Nanning, Guangxi, 530004, China;2.College of Digital Economics, Nanning University, Nanning, Guangxi, 530299, China) |
| Abstract: |
| Traditional word vector embedding models,such as Word2Vec and GloVe,cannot realize polysemy expression.Traditional text classification models also fail to make good use of the semantic information of label words.Based on this,a classification model based on XLNet+BiGRU+Att (Label) is proposed.Firstly,the dynamic word vector expression of text sequence and label sequence is generated by XLNet.Then,the text vector is input into the Bidirectional Gated Recurrent Unit (BiGRU) to extract the text feature information.At last,the label words are combined with the attention mechanism to select the tendency label words of the text,calculate the attention score of the tendency label words and the text vector,and update the text vector according to the attention score.Through comparative experiments,the model in this paper has higher accuracy than the traditional model in text classification tasks.Using XLNet as the word embedding model and combining label words in attention calculation can improve the classification performance of the model. |
| Key words: text classification XLNet BiGRU label word attention mechanism |
