Article Details

Title · Authors · Abstract · Keywords · DOI · References

Research on an Automatic Annotation Method of Automotive Fault Records Based on Large Language Models

DAI Yawen, ZHAO Yue, ZHANG Lin

(Wuhu Institute of Technology, Wuhu 241000, China)

Section:Computer Technology · 2025, Vol. 9, No. 12 · Pages:92-96 · 2096-4706(2025)12-0092-05 · CLC number:TP391.1 · Document code:A · DOI:10.19850/j.cnki.2096-4706.2025.12.018

Abstract

Automotive fault records are the crucial data source for automotive condition analysis. However, the unstructured nature and technical specificity make the cost of manual annotation high. To address this issue, this paper proposes an automatic annotation method based on Large Language Models. The study combines LLMs with domain-specific knowledge by constructing special prompt templates to extract and annotate key data, such as faulty parts, fault status, and diagnostic tools. Experimental results show that the DeepSeek model, using a Few-shot Prompting template combined with industry examples, achieves the best performance, with an annotation precision of 79.65%, recall of 84.92%, and F1 score of 82.20%. This outperforms traditional Deep Learning-based annotation methods by more than 3% in terms of F1 score.

Keywords

Large language Model; automotive fault record; automatic annotation; Prompt Engineering

References

  1. [1] Chen Jingbing Research and Implementation of Network Threat Intelligence Information Extraction Method for Unstructured Text [D]. Beijing: Beijing University of Posts and Telecommunications, 2024
  2. [2] Ren Anqi, Liu Lin, Wang Hailong, et al. A Review of Research on Text Entity Relationship Extraction [J]. Computer Science and Exploration, 2024, 18 (11): 2848-2871
  3. [3] Hu Hangle, Cheng Chunlei, Ye Qing, et al. A Review of Open Information Extraction Research [J]. Computer Engineering and Applications, 2023, 59 (16): 31-49
  4. [4] Zhu Xiaolong, Qiu Lin Simulation of semi-structured text information extraction in machine learning [J]. Computer Simulation, 2023, 40 (2): 540-544
  5. [5] ZHAO L J,QIU X P,ZHANG Q,et al. Sequence Labeling With Deep Gated Dual Path CNN [J].IEEE/ACM Transactions on Audio, Speech, and Language Processing,2019,27(12):2326-2335.
  6. [6] ALZAIDY R,CARAGEA C,GILES C L. Bi-LSTM-CRF Sequence Labeling for Keyphrase Extraction from Scholarly Documents [C]//The World Wide Web Conference.San Francisco:ACM,2019:2551-2557.
  7. [7] Sun Zhengyan, Zhang Shunxiang, Chen Lei, et al. Causal relationship extraction model based on BERT semantic enhancement [J]. Computer Engineering and Design, 2024, 45 (12): 3758-3763
  8. [8] Huang Rui, Zhao Changming, Li Hanqing, et al. Intelligent driving data automatic annotation method based on visual big model [C]//31st SAE China Annual Conference and Exhibition (SAECCE 2024). Chongqing: SAE China, 2024: 263-270
  9. [9] DEEPSEEK-AI,GUO D Y,YANG D J,et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning [J/OL].arXiv:2501.12948 [cs.CL].(2025-01-22).https://arxiv.org/abs/2501.12948.
  10. [10] SUN Y,WANG S H,FENG S K,et al. ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation [J/OL].arXiv:2107.02137 [cs.CL].(2021-07-05).https://arxiv.org/abs/2107.02137.
  11. [11] QIN R Y,LI Z M,HE W R,et al. Mooncake: A KVCache-centric Disaggregated Architecture for LLM serving [J/OL].arXiv:2407.00079 [cs.DC].(2024-06-24).https://arxiv.org/abs/2407.00079.
  12. [12] Chen Wentai, Zhang Fan Research on the influence of prompt word constituent elements on the quality of cross modal content generation in large language models: an exploratory experiment based on the text generation graph function of the iFlytek Spark large model [J]. Journal of Zhengzhou University: Philosophy and Social Sciences Edition, 2025, 58 (2): 99-104
  13. [13] RUBIN O,HERZIG J,BERANT J. Learning to Retrieve Prompts for In-context Learning [J/OL].arXiv:2112.08633 [cs.CL].(2021-12-16).https://arxiv.org/abs/2112.08633.

Funding

Key Project of Natural Science Research in Universities of Anhui Province (2023AH052378); Key Project of Natural Science Research in Universities of Anhui Province (2023AH052392); Wuhu Vocational and Technical College level Research Project (wzyzrzd202405); Key Project of Natural Science Research in Universities of Anhui Province (2022AH052211); Key Project of Natural Science Research in Universities of Anhui Province (2022AH052198); Anhui Provincial Teaching Innovation Team (2023cxtd147); Anhui Province Curriculum Ideological and Political Demonstration Course (2023kcsz004); Wuhu Vocational and Technical College University level Research Project (wzyzrzd202402)

About the authors

Dai Yawen (1997-), female, Han nationality, from Wuhu, Anhui Province, teaching assistant, master's student, research direction: natural language processing; Zhao Yue (1996-), female, Han ethnicity, from Suzhou, Anhui Province, teaching assistant, master's student, research direction: image encryption; Zhang Lin (1984-), female, Han ethnicity, from Xuancheng, Anhui Province, associate professor, master's student, research direction: computer application technology.