基于机器阅读理解的中文司法实体识别优化策略研究

Research on Optimization Strategy of Chinese Judicial Entity Recognition Based on Machine Reading Comprehension

关键词:  司法信息抽取|预训练|自注意力机制|标签语义|对抗训练

Key words:  judicial information extraction|pre-training|self-attention mechanism|label semantics|adversarial training

摘要:

针对中文司法领域信息抽取数据集中实体专业性较强、现有机器阅读理解(MRC)模型无法通过构建问句提供充足的标签语义且在噪声样本上表现不佳等问题，本研究提出了一种联合优化策略。首先，通过聚合在司法语料中多次出现的实体构建司法领域词典，将专业性较强的实体知识注入RoBERTa-wwm预训练语言模型进行预训练。然后，通过基于自注意力机制来区分每个字对不同标签词的重要性，从而将实体标签语义融合到句子表示中。最后，在微调阶段采用对抗训练算法对模型进行优化，增强模型的鲁棒性和泛化能力。在2021年中国法律智能评测(CAIL2021)司法信息抽取数据集上的实验结果表明：相较于基线模型，本研究方法F1值提高了2.79%，并且模型在CAIL2021司法信息抽取赛道中获得了全国三等奖的成绩，验证了联合优化策略的有效性。

Abstract:

Aiming at the problems that the entities in the Chinese judicial information extraction dataset are highly professional,the existing Machine Reading Comprehension (MRC) model cannot provide sufficient label semantics by constructing questions and performs poorly on noise samples,a joint optimization strategy is proposed in this study.Firstly,a judicial domain dictionary is constructed by aggregating entities that appear many times in the judicial corpus,and professional entity knowledge is injected into the RoBERTa-wwm pre-training language model for pre-training.Then,the entity label semantics are integrated into the sentence representation by distinguishing the importance of each word to different label words based on the self-attention mechanism.Finally,in the fine-tuning stage,the adversarial training algorithm is used to optimize the model to enhance the robustness and generalization ability of the model.The experimental results on the 2021 China Legal Intelligence Evaluation (CAIL2021) judicial information extraction dataset show that compared with the baseline model,the F1 value of this research method is increased by 2.79%.And the model in the CAIL2021 judicial information extraction track won the national third prize,which verified the effectiveness of the joint optimization strategy.