Article Details

Title · Authors · Abstract · Keywords · DOI · References

A Training and Value Alignment Method for Multimodal Large Models in the Video Surveillance Industry

LIU Yang, NING Mingtao

(CRSC Communication and Information Group Co., Ltd., Beijing 100070, China)

Section：Artificial Intelligence · Vol. 2026, Issue 9 · Pages：26-30,39 · 2096-4706（2026）09-0026-06 · CLC number：TP18 · Document code：A · DOI：10.19850/j.cnki.2096-4706.2026.09.006

Abstract

With the growing adoption of Multimodal Large Models (MLLMs) in industry applications, balancing generalization with domain-specific value alignment has become a pressing challenge. This paper proposes a multimodal training and value-alignment framework tailored for the surveillance industry. The method introduces industry-specific paired images at the visual input stage to jointly capture global semantics and local details, integrates task-oriented system prompts with “reasoning first” and “deciding first” strategies during training, and applies reinforcement learning based on domain preferences for dynamic optimization. Experimental results show that our approach achieves an average score of 71.4 across 10 benchmarks, outperforming models of similar scale by 3.3～4.8 points (approximately 4.8%～7.2%). On the industry dataset GHengVision, it attains 70.4, significantly exceeding general-purpose baselines. These results demonstrate the proposed method's effectiveness in enhancing interpretability, reasoning consistency, and domain-value alignment for practical multimodal applications.

Keywords

Multimodal Large Model; industry data-driven; visual-semantic alignment; prompt optimization; Reinforcement Learning

References

  1. [1] ALAYRAC J-B，DONAHUE J，LUC P，et al. Flamingo: A Visual Language Model for Few-shot Learning [J/OL].arXiv:2204.14198 [cs.CV].（2022-04-29）.https://arxiv.org/abs/2204.14198v2.
  2. [2] LI J N，LI D X，XIONG C M，et al. BLIP: Bootstrapping Language-image Pre-training for Unified Vision-Language Understanding and Generation [J/OL].arXiv:2201.12086 [cs.CV].（2022-01-28）.https://arxiv.org/abs/2201.12086.
  3. [3] LI J N，LI D X，SAVARESE S，et al. BLIP-2: Bootstrapping Language-image Pre-training with Frozen Image Encoders and Large Language Models [J/OL].arXiv:2301.12597 [cs.CV].（2023-01-30）.https://arxiv.org/abs/2301.12597.
  4. [4]ZHU D Y ，CHEN J ，SHEN X Q ，et al. MiniGPT-4: Enhancing Vision-language Understanding with Advanced Large Language Models [J/OL].arXiv:2304.10592 [cs.CV].（2023-04-20）.https://arxiv.org/abs/2304.10592.
  5. [5] BAI J Z ，BAI S，YANG S S，et al. Qwen-VL: A Versatile Vision-language Model for Understanding, Localization, Text Reading, and Beyond [J/OL].arXiv:2308.12966 [cs.CV].（2023-08-24）.https://arxiv.org/abs/2308.12966.
  6. [6] WANG W Y，GAO Z W，GU L X，et al. InternVL3.5: Advancing Open-source Multimodal Models in Versatility, Reasoning, and Efficiency [J/OL].arXiv:2508.18265 [cs.CV].（2025-08-25）.https://arxiv.org/abs/2508.18265.
  7. [7] SILVA U D，FERNANDO L，LIK B L P，et al. Large Language Models for Video Surveillance Applications [J/OL].arXiv:2501.02850 [cs.CV].（2025-01-06）.https://arxiv.org/abs/2501.02850.
  8. [8] GONZALEZ J A A，MATSUKAWA T，SUZUKI E. Leveraging Vision Language Models for Understanding and Detecting Violence in Videos [C]//20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications.Porto：SciTePress，2025，2：99-113.
  9. [9] CHEN L，LI J S，DONG X Y，et al. Are We on the Right Way for Evaluating Large Vision-Language Models? [J/OL].arXiv:2403.20330 [cs.CV].（2024-03-29）.https://arxiv.org/abs/2403.20330.
  10. [10] YU W H，YANG Z Y，LI L J，et al. MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities [J/OL].arXiv:2308.02490 [cs.AI].（2023-08-04）.https://arxiv.org/abs/2308.02490.
  11. [11] LIU Y，DUAN H D，ZHANG Y H，et al. MMBench: Is Your Multi-modal Model an All-around Player? [J/OL].arXiv:2307.06281 [cs.CV].（2023-07-12）.https://arxiv.org/abs/2307.06281.
  12. [12] TANG J Q，LIU Q，YE Y J，et al. MTVQA: Benchmarking Multilingual Text-centric Visual Question Answering [C]//Findings of the Association for Computational Linguistics.Vienna：ACL，2025：7748-7763.
  13. [13] KEMBHAVI A，SALVATO M，KOLVE E，et al. A Diagram is Worth a Dozen Images [J/OL].arXiv:1603.07396 [cs.CV].（2016-03-24）.https://arxiv.org/abs/1603.07396.
  14. [14] LIU Y L，LI Z，HUANG M X，et al. OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models [J/OL].Science China Information Sciences，2024，67（12）：220102（2024-12-11）[2025-09-11].https://link.springer.com/article/10.1007/s11432-024-4235-6.
  15. [15] LU Y J，JIANG D F，CHEN W H，et al. WildVision: Evaluating Vision-language Models in the Wild with Human Preferences [J/OL].arXiv:2406.11069 [cs.CV].（2024-06-16）.https://arxiv.org/abs/2406.11069.
  16. [16] ZHANG Y F，ZHANG H Y，TIAN H C，et al. MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans? [J/OL].arXiv:2408.13257 [cs.CV].（2024-08-23）.https://arxiv.org/abs/2408.13257.
  17. [17] GUAN T R，LIU F X，WU X Y，et al. HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models [J/OL].arXiv:2310.14566 [cs.CV].（2023-10-23）.https://arxiv.org/abs/2310.14566.

About the authors

Liu Yang (born on October 1983), male, Han ethnicity, from Kaiyuan, Liaoning, is a senior engineer with a PhD. His research interests include artificial intelligence and computer vision; Ning Mingtao (born on August 10, 1978), male, Han ethnicity, from Cangzhou, Hebei Province, is a senior engineer with a bachelor's degree. His research interests include artificial intelligence applications and big data analysis.