Article Details

Abstract

With the widespread application of intelligent question answering and retrieval systems in scenarios such as enterprise knowledge management, academic research, and professional field question answering, ensuring their quality has become a critical issue that urgently needs to be addressed. Unlike traditional software, intelligent question answering systems exhibit core AI characteristics such as uncertainty, ambiguity tolerance, context dependency, blurred knowledge boundaries, and security vulnerability, making it difficult for traditional test case design methods to effectively evaluate their quality. This paper provides an in-depth analysis of the AI characteristics of intelligent question answering systems and the challenges they pose to testing. It proposes a set of functional test case design methods tailored to the characteristics of AI systems. Taking the CNKI AI knowledge question answering function as a case study, the paper elaborates on the design practice of functional test cases targeting five major AI characteristics, including uncertainty testing based on metamorphic testing, robustness testing based on text variation, context testing based on dialog graph traversal, knowledge boundary testing based on boundary detection, and security testing based on adversarial attacks. The practical results show that this method successfully executes 200 test cases and identifies 33 issues (including 2 severe defects), effectively enhancing the quality and stability of the system. This paper provides systematic methodological guidance and reusable engineering practice references for the testing of intelligent question answering systems.

Keywords

AI characteristics; test case design; intelligent question answering; quality assurance

References

1. [1] Guo Qingxiang, Bao Liangliang, Zhou Huacong, etc Application Practice of CNKI AI in Medicinal Chemistry Teaching [J]. Journal of Higher Education, 2026, 12 (3): 114-117
2. [2] Huazhi Big Model 2.0 officially released in Shenzhen [J]. China Military to Civilian Conversion, 2024 (9): 8
3. [3] Ding Shaobin, Wang Hongju Artificial Intelligence and Financial Security: Technological Vulnerability, Risk Evolution Path, and Prevention Mechanism [J]. Economic System Reform, 2026 (2): 27-36
4. [4] Su Zhenhua, Shu Le Within Risk, Beyond Uncertainty: The Trust Boundary of Generative AI in Public Communication [J]. Young Journalist, 2025 (12): 20-26, 32
5. [5] Shen Qingchao, Li Xingjian, Jiang Jiajun, etc Intelligent Q&A System Logical Reasoning Test [J]. Journal of Software, 2026, 37 (2): 543-562
6. [6] Ru Bingsheng, Chen Lei, Zhu Shaomin Authoritative Guide to Modern Software Testing Techniques [M]. Beijing: Electronic Industry Press, 2025: 856
7. [7] DB21/T 3897-2023: Zhang Jiannan Software engineering software defect level determination method [S]
8. [8] Lu Hui, Wang Shiqi, Wan Xiong, etc Research on Verification Case Generation Method for Robustness of Power Grid Dispatching Automation System [J]. Computer Measurement and Control, 2026, 34 (1): 259-268
9. [9] Zhou Hui, Guo Hongyou Technical Governance of Large Language Model Security: Adversarial Testing and Evaluation Audit [J]. Journal of Xi'an Jiaotong University (Social Sciences Edition), 2025, 45 (2): 78-88
10. [10] Zheng Zhen, Deng Bojun Research on Indicators and Methods for Ethical Risk Assessment of Artificial Intelligence [J]. Artificial Intelligence, 2025 (2): 80-92

About the authors

Liu Xiaoling (August 1980-), female, Han ethnicity, from Fengxian, Jiangsu, senior engineer, bachelor's degree, research interests include software test management.