Research on Collaborative Proofreading Method of Large Language Model Based on BERT Model and RLHF
(1.State Grid Hubei Electric Power Co., Ltd., Wuhan 430048, China; 2.Wuhan Optics Valley Information Technology Co., Ltd., Wuhan 430206, China)
Abstract
The accuracy of document proofreading has always faces challenges at the level of complex logic. In order to alleviate the pressure on writers and front-line staff, this study proposes a proofreading method based on multi-model collaboration. The word-by-word label is generated by fine-tuning BERT model, and the Large Language Model is fine-tuning using LoRA to compensate for deficiencies in deep error understanding. The PPO algorithm is used to optimize the decision-making process of the model to meet the needs of different scenarios. The multi-model output results are integrated through XGBoost to avoid underreporting and misreporting. The experimental results show that this method has significant advantages in improving the quality and accuracy of document proofreading.
Keywords
document proofreading; BERT; LLM; PPO; XGBoost
References
- [1] Hu Yuande Following the example of a gourd and drawing a gourd ": Format Norms and Content Patterns in Official Document Writing [J]. News and Writing, 2023 (9): 106-108
- [2] Feng Qingyun Several Thoughts on Official Document Writing [J]. Secretarial Work, 2024 (9): 54-55
- [3] ELMAN J L. Finding Structure in Time [J].Cognitive Science,1990,14(2):179-211.
- [4] DEVLIN J,CHANG M W,LEE K,et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [J/OL].arXiv:1810.04805 [cs.CL].[2024-12-20].https://doi.org/10.48550/arXiv.1810.04805.
- [5] YU C,VELU A,VINITSKY E,et al. The Surprising Effectiveness of PPO in Cooperative Multi-agent Games [C]//Proceedings of the 36th International Conference on Neural Information Processing Systems.New Orleans:Curran Associates Inc.,2022:24611-24624.
- [6] Zhang Zhuo A deep learning based automatic proofreading system for official documents [J]. Software, 2020, 41 (11): 94-96+105
- [7] LI Y Y,KRISHNAMURTHY R,RAGHAVAN S,et al. Regular Expression Learning for Information Extraction [C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing.Honolulu:Association for Computational Linguistics,2008:21-30.
- [8] KOROTEEV M V. BERT: A Review of Applications in Natural Language Processing and Understanding [J/OL].arXiv:2103.11943 [cs.CL].[2024-12-15].https://doi.org/10.48550/arXiv.2103.11943.
- [9] HAN K,XIAO A,WU E H,et al. Transformer in Transformer [J/OL]. arXiv:2103.00112 [cs.CV].[2024-12-19].https://doi.org/10.48550/arXiv.2103.00112.
- [10] YANG A,YANG B S,ZHANG B C,et al. Qwen2.5 Technical Report [J/OL].arXiv:2412.15115 [cs.CL].[2024-11-23].https://doi.org/10.48550/arXiv.2412.15115.
- [11] YAO Y F,DUAN J H,XU K D,et al. A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly [J/OL].High-Confidence Computing,2024,4(2):100211[2024-11-27].https://doi.org/10.1016/j.hcc.2024.100211.
- [12] ZHENG R,DOU S H,GAO S Y,et al. Secrets of RLHF in Large Language Models Part I: PPO [J/OL].arXiv:2307.04964 [cs.CL].[2024-10-06].https://doi.org/10.48550/arXiv.2307.04964.
- [13] YU C,VELU A,VINITSKY E,et al. The Surprising Effectiveness of PPO in Cooperative Multi-agent Games [C]//Proceedings of the 36th International Conference on Neural Information Processing Systems.New Orleans:Curran Associates Inc.,2022,35:24611-24624.
- [14] CHEN T Q,GUESTRIN C. XGBoost: A Scalable Tree Boosting System [C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.New York:ACM,2016:785-794.
- [15] CHEN T,TONG H,BENESTY M,et al. xgboost: Extreme Gradient Boosting [EB/OL].[2024-11-17].https://github.com/dmlc/xgboost.
About the authors
Wu Bian (1981-), male, Han ethnicity, from Jingmen, Hubei, is a power engineering engineer and a master's student. His research interests include office automation digitization and secretarial work; Yang Zhengtan (1999-), male, Han ethnicity, from Linzhou, Henan Province, is a junior algorithm engineer with a bachelor's degree. His research direction is artificial intelligence and applications; Li Xiang (1991-), male, Han ethnicity, from Wuhan, Hubei, is a senior algorithm engineer and a master's student. His research direction is artificial intelligence models and algorithms.