Article Details

Abstract

The spelling correction method for Traditional Chinese Medicine (TCM) texts aims to identify and correct spelling errors in unstructured TCM texts. High-quality TCM texts facilitate the application and research of downstream tasks. Traditional spelling correction models lack the ability to deeply fuse and extract text features, failing to fully explore and understand contextual information. To address the above issues, a spelling correction model based on Attention Mechanism and multi-feature fusion, named TCM-CSC, is proposed to enhance the correction capability. TCM-CSC is trained on the ChineseBert pre-trained model, and a syllable information classifier with a multi-head Attention Mechanism is added on top of its character, pinyin, and glyph embedding layers. This classifier extracts syllable feature information and integrates it into the correction process as an auxiliary signal, enabling it to better capture the association between syllables and characters. Experimental results show that TCM-CSC precision and F1 score reached 77.3% and 81.99%, respectively, in the spelling correction task of TCM texts, significantly improving the model's correction performance.

Keywords

Traditional Chinese Medicine text; spelling correction; Attention Mechanism; multi-feature fusion

References

1. [1] LIU L F，WU H Q，ZHAO H. Chinese Spelling Correction as Rephrasing Language Model [C]//Proceedings of the 38th AAAI Conference on Artificial Intelligence.Vancouver：AAAI Press，2024，38（17）：18662-18670.
2. [2] YU J J，LI Z H. Chinese Spelling Error Detection and Correction Based on Language Model, Pronunciation, and Shape [C]//Proceedings of the Third CIPS-SIGHAN Joint Conference on Chinese Language Processing.Wuhan：ACL，2021：220-223.
3. [3] LI F F，JIANG J，TANG D F，et al. Efficient Word Segmentation for Enhancing Chinese Spelling Check in Pre-trained Language Model [J].Knowledge and Information Systems，2025，67（1）：603-632.
4. [4] VASWANI A，SHAZEER N，PARMAR N，et al. Attention is All You Need [J/OL].arXiv:1706.03762 [cs.CL].（2017-06-12）.https://arxiv.org/abs/1706.03762?ref=andrealarosa.org.
5. [5] DEVLIN J，CHANG M-W，LEE K，et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.Minneapolis：ACL，2019，1：4171-4186.
6. [6] HONG Y Z，YU X G，HE N，et al. FASPell: A Fast, Adaptable, Simple, Powerful Chinese Spell Checker Based on DAE-decoder Paradigm [C]//Proceedings of the 5th Workshop on Noisy User-generated Text(W-NUT 2019).Hong Kong：ACL，2019：160-169.
7. [7] ZHANG S H，HUANG H R，LIU J C，et al. Spelling Error Correction with Soft-Masked BERT [J/OL].arXiv:2005.07421 [cs.CL].（2020-05-15）.https://arxiv.org/abs/2005.07421?context=cs.LG.
8. [8] CHENG X Y，XU W D，CHEN K L，et al. SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check [J/OL].arXiv:2004.14166 [cs.CL].（2020-04-26）.https://arxiv.org/abs/2004.14166.
9. [9] LIU S L，YANG T，YUE T C，et al. PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction [C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing.Online：ACL，2021，1：2991–3000.
10. [10] HUANG L，LI J J，JIANG W W，et al. PHMOSpell: Phonological and Morphological Knowledge Guided Chinese Spelling Check [C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing.online：ACL，2021：5958-5967.
11. [11] LIU C L，LAI M-H，TIEN K-W，et al. Visually and Phonologically Similar Characters in Incorrect Chinese Words: Analyses, Identification, and Applications [J].ACM Transactions on Asian Language Information Processing，2011，10（2）：10-39.
12. [12] LI J H，WANG Q，MAO Z D，et al. Improving Chinese Spelling Check by Character Pronunciation Prediction: The Effects of Adaptivity and Granularity [C]//Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing.Abu Dhabi：ACL，2022：4275-4286.
13. [13] YUAN H Q，HE Y，DU P，et al. Multi-task Learning Using Uncertainty to Weigh Losses for Heterogeneous Face Attribute Estimation [J/OL].arXiv:2403.00561 [cs.CV].（2024-03-01）.https://arxiv.org/abs/2403.00561.
14. [14] CHENG X Y，XU W D，CHEN K L，et al. SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check [J/OL].arXiv:2004.14166 [cs.CL].（2020-04-26）.https://arxiv.org/abs/2004.14166.
15. [15] XU H-D，LI Z L，ZHOU Q Y，et al. Read, Listen, and See: Leveraging Multimodal Information Helps Chinese Spell Checking [C]//Findings of the Association for Computational Linguistics, ACL-IJCNLP 2021.Online：ACL，2021：716-728.

Funding

Natural Science Foundation of Jiangxi Province (20224BAB206102); National Natural Science Foundation of China (82260988); Jiangxi Provincial Health and Family Planning Commission Science and Technology Plan Project (202211404) Jiangxi Provincial Department of Education Science and Technology Research Project (GJJ2200923)

About the authors

Shen Youzhi (1997-), male, Han ethnicity, from Jiujiang, Jiangxi Province, is currently pursuing a master's degree in natural language processing; Corresponding author: Cheng Chunlei (1976-), male, Han ethnicity, from Nanchang, Jiangxi Province, associate professor, master's supervisor, PhD, research directions: machine learning, knowledge representation and learning, knowledge graph; Ju Zedong (1998-), male, Han ethnicity, from Shuozhou, Shanxi Province, currently pursuing a master's degree in natural language processing; Gong Zhufan (2000-), male, Han ethnicity, from Nanchang, Jiangxi, currently pursuing a master's degree in natural language processing.