基于BERT模型与RLHF的大语言模型协同校对方法研究
(1.国网湖北省电力有限公司,湖北 武汉 430048;2.武汉光谷信息技术股份有限公司,湖北 武汉 430206)
摘要
公文校对的准确性在复杂逻辑层面一直面临挑战。为缓解写作人员与一线工作人员的压力,该研究提出一种基于多模型协同的校对方法。通过微调BERT模型生成逐字标签,结合LoRA微调大语言模型,弥补深层次错误理解的不足。利用PPO算法优化模型决策过程,使其适应不同场景需求。将多模型输出结果通过XGBoost集成,避免漏报与错报现象。实验结果表明,该方法在提升公文校对质量和准确性方面具有显著优势。
关键词
公文校对;BERT;LLM;PPO;XGBoost
参考文献
- [1] 胡元德.“依葫芦画瓢”:公文写作的格式规范与内容模式 [J].新闻与写作,2023(9):106-108.
- [2] 封清云.对公文写作的几点思考 [J].秘书工作,2024(9):54-55.
- [3] ELMAN J L. Finding Structure in Time [J].Cognitive Science,1990,14(2):179-211.
- [4] DEVLIN J,CHANG M W,LEE K,et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [J/OL].arXiv:1810.04805 [cs.CL].[2024-12-20].https://doi.org/10.48550/arXiv.1810.04805.
- [5] YU C,VELU A,VINITSKY E,et al. The Surprising Effectiveness of PPO in Cooperative Multi-agent Games [C]//Proceedings of the 36th International Conference on Neural Information Processing Systems.New Orleans:Curran Associates Inc.,2022:24611-24624.
- [6] 张卓.基于深度学习的公文自动校对系统 [J].软件,2020,41(11):94-96+105.
- [7] LI Y Y,KRISHNAMURTHY R,RAGHAVAN S,et al. Regular Expression Learning for Information Extraction [C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing.Honolulu:Association for Computational Linguistics,2008:21-30.
- [8] KOROTEEV M V. BERT: A Review of Applications in Natural Language Processing and Understanding [J/OL].arXiv:2103.11943 [cs.CL].[2024-12-15].https://doi.org/10.48550/arXiv.2103.11943.
- [9] HAN K,XIAO A,WU E H,et al. Transformer in Transformer [J/OL]. arXiv:2103.00112 [cs.CV].[2024-12-19].https://doi.org/10.48550/arXiv.2103.00112.
- [10] YANG A,YANG B S,ZHANG B C,et al. Qwen2.5 Technical Report [J/OL].arXiv:2412.15115 [cs.CL].[2024-11-23].https://doi.org/10.48550/arXiv.2412.15115.
- [11] YAO Y F,DUAN J H,XU K D,et al. A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly [J/OL].High-Confidence Computing,2024,4(2):100211[2024-11-27].https://doi.org/10.1016/j.hcc.2024.100211.
- [12] ZHENG R,DOU S H,GAO S Y,et al. Secrets of RLHF in Large Language Models Part I: PPO [J/OL].arXiv:2307.04964 [cs.CL].[2024-10-06].https://doi.org/10.48550/arXiv.2307.04964.
- [13] YU C,VELU A,VINITSKY E,et al. The Surprising Effectiveness of PPO in Cooperative Multi-agent Games [C]//Proceedings of the 36th International Conference on Neural Information Processing Systems.New Orleans:Curran Associates Inc.,2022,35:24611-24624.
- [14] CHEN T Q,GUESTRIN C. XGBoost: A Scalable Tree Boosting System [C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York:ACM,2016:785-794.
- [15] CHEN T,TONG H,BENESTY M,et al. xgboost: Extreme Gradient Boosting [EB/OL].[2024-11-17].https://github.com/dmlc/xgboost.
作者简介
吴边(1981—),男,汉族,湖北荆门人,电力工程工程师,硕士研究生,研究方向:办公自动化数字化以及文秘工作;杨政潭(1999—),男,汉族,河南林州人,初级算法工程师,本科,研究方向:人工智能与应用;李翔(1991—),男,汉族,湖北武汉人,高级算法工程师,硕士研究生,研究方向:人工智能模型与算法。