Article Details

Title · Authors · Abstract · Keywords · DOI · References

Recognition Method for Handwritten Chemical Structures Based on Multi-model Fusion

LI Yuanjun, CHEN Jianhu, ZHANG Yabiao, ZHANG Bin, XU Qian, ZHAO Yifang, BI Yanhong, CHEN Lin, SHI Qinghui, XUE Dejun

(Tongfang CNKI Digital Technology Co., Ltd., Beijing 100192, China)

Section：Artificial Intelligence · Vol. 2026, Issue 1 · Pages：58-62,67 · 2096-4706（2026）01-0058-06 · CLC number：TP391.4 · Document code：A · DOI：10.19850/j.cnki.2096-4706.2026.01.012

Abstract

Current chemical structure recognition methods mostly rely on printed chemical images and are sensitive to handwritten perturbations and image quality fluctuations during the structural decoding process. Especially when images exhibit blur, deformation, or background interference, deep models easily output illegal or incomplete structural results, and the output consistency among different training models is poor, which further limits the application of the model in actual complex environments. To address this problem, this paper proposes an end-to-end chemical structure recognition method (VoteOCSR) combining structural awareness and a robust fusion mechanism. It consists of three parts, including structural formula pre-training, data augmentation-driven multi-model generation, and structural legality-guided voting fusion, and it can stably output legal structural codes under diverse input conditions. Experimental results show that it achieves an Exact Match (EM) structural accuracy of 70.66% on the ICDAR 2024 competition dataset, which is in a leading position, and all indicators are superior to baseline methods.

Keywords

chemical structure recognition; data augmentation; fusion mechanism; ICDAR

References

  1. [1] KRASNOV A，BARNABAS S J，BOEHME T，et al. Comparing Software Tools for Optical Chemical Structure Recognition [J].Digital Discovery，2024，3（4）：681-693.
  2. [2] Xue Zijia Research on Chemical Structure Recognition Method of Small Molecule Drugs Based on Transform Neural Network [D]. Chengdu: Xihua University, 2024
  3. [3] STAKER J，MARSHALL K，ABEL R，et al. Molecular Structure Extraction from Documents Using Deep Learning [J].Journal of Chemical Information and Modeling，2019，59（3）：1017-1029.
  4. [4] KHOKHLOV I，KRASNOV L，FEDOROV M，et al. Image2SMILES: Transformer-based Molecular Optical Recognition Engine [J].ChemRxiv，2022，2（1）：1-24.
  5. [5] Lin Fan, Li Jianhua Optical Chemical Structure Recognition Based on Multi order Gated Aggregation Network [J]. Computer Engineering, 2025, 51 (8): 364-372
  6. [6] Zeng Shuiling, Li Zhaoxian, Zhang Jiaxiong, etc Chemical structure recognition method combining attention mechanism and encoder decoder architecture [J]. Chinese Journal of Image and Graphics, 2024, 29 (7): 1960-1969
  7. [7] BLECHER L，CUCURULL G，SCIALOM T，et al. Nougat: Neural Optical Understanding for Academic Documents [J/OL].arXiv:2308.13418 [cs.LG].[2025-06-10].https://doi.org/10.48550/arXiv.2308.13418.
  8. [8] PINASTHIKA K，LAKSONO B S P，IRSAL R B P，et al. Sparseswin: Swin Transformer with Sparse Transformer Block [J/OL].Neurocomputing，2024，580：127433[2025-06-03].https://doi.org/10.1016/j.neucom.2024.127433.
  9. [9] ZHAO X Y，HU T，MAO C X，et al. HAT: A Visual Transformer Model for Image Recognition Based on Hierarchical Attention Transformation [J].IEEE Access，2023，11：100042-100051.
  10. [10] CHIPMAN H A，GEORGE E I，MCCULLOCH R E，et al. mBART: Multidimensional Monotone BART [J/OL].arXiv:1612.01619 [stat.OT].[2025-07-01].https://doi.org/10.48550/arXiv.1612.01619.
  11. [11] XU Z，LI J，YANG Z，et al. SwinOCSR: End-to-End Optical Chemical Structure Recognition Using a Swin Transformer [J/OL].Journal of Cheminformatics，2022，14（1）：41[2025-06-21].https://doi.org/10.1186/s13321-022-00624-5.

About the authors

Li Yuanjun (1992-), male, Han ethnicity, from Guyuan, Ningxia, project manager, bachelor's degree, research interests include computer vision, deep learning and document digitalization; Corresponding author: Chen Jianhu (1992-), male, Han ethnicity, from Lanzhou, Gansu, deep learning engineer, master's student, research interests include computer vision, deep learning and machine learning.