Article Details

Abstract

To address the issues of high computational complexity and deployment costs in classroom behavior recognition, this paper proposes a lightweight behavior recognition model optimization method based on knowledge distillation. First, ResNet-50 is employed as the teacher model to extract deep spatiotemporal features from classroom videos, and these features are transferred to the lightweight student model MobileNetV3-small via knowledge distillation. Second, a hybrid loss function fusing KL divergence and attention alignment is designed to enhance the student model's ability to imitate key spatiotemporal features from the teacher model. Finally, a subset for educational scenario behavior recognition is constructed based on the public dataset SCB-Dataset3 to validate the model's performance. Experimental results demonstrate that the optimized student model achieves a 78% reduction in parameters and a 3.2-fold improvement in inference speed while incurring only a 1.7% accuracy drop compared with the teacher model, significantly outperforming traditional pruning and quantization methods. This provides a practical technical solution for lightweight behavior recognition in educational scenarios.

Keywords

knowledge distillation; classroom behavior recognition; model compression; MobileNetV3-small; hybrid distillation

References

1. [1] Bi Feilong Knowledge Distillation Based on Training Teachers and Comparative Model Sequence [D]. Huainan: Anhui University of Science and Technology, 2024
2. [2] Wei Yantao, Qin Daoying, Hu Jiamin, etc Student Classroom Behavior Recognition Based on Deep Learning [J]. Modern Educational Technology, 2019, 29 (7): 87-91
3. [3] Targ S，Almeida D，Lyman K. Resnet in Resnet: Generalizing Residual Architectures [J/OL].arXiv:1603.08029 [cs.LG].https://doi.org/10.48550/arXiv.1603.08029.
4. [4] Zhu Yunpeng, Huang Xi, Huang Jiaxing Research on Human Action Recognition Based on 3D CNN [J]. Modern Electronic Technology, 2020, 43 (18): 150-152156
5. [5] Gao Han, Tian Yulong, Xu Fengyuan, etc A review of deep learning model compression and acceleration [J]. Journal of Software, 2021, 32 (1): 68-92
6. [6] Li Ruoyu, Yun Lijun, Jin Xuesong, etc Gait recognition method based on knowledge distillation [J]. Journal of Yunnan Normal University (Natural Science Edition), 2024, 44 (2): 49-53
7. [7] Wang Nan, Wang Qi, Ouyang Dantong A multimodal sentiment analysis model based on knowledge distillation and dynamic adjustment mechanism [J]. Journal of Computer Science, 2025, 48 (8): 1923-1942
8. [8] Yao Yi, Guo Zhilin Abnormal behavior recognition in examination rooms based on knowledge distillation [J]. Computer Applications and Software, 2025, 42 (3): 156-161
9. [9] Li Ruoyu, Yun Lijun, Jin Xuesong, etc Gait recognition method based on knowledge distillation [J]. Journal of Yunnan Normal University (Natural Science Edition), 2024, 44 (2): 49-53
10. [10] Huang Mingjie Research on Plagiarism Detection Technology Based on Knowledge Distillation [D]. Foshan: Foshan University of Science and Technology, 2024
11. [11] Hinton G E，Vinyals O，Dean J. Distilling the Knowledge in a Neural Network [J/OL].arXiv:1503.02531 [stat.ML].https://doi.org/10.48550/arXiv.1503.02531.
12. [12] Gou J，Yu B，Maybank S J，et al. Knowledge Distillation: A Survey [J].International Journal of Computer Vision，2021，129（6）：1789-1819.
13. [13] Howard A，Sandler M，Chen B，et al. Searching for MobileNetV3 [C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV)，2020：1314-1324.
14. [14] Yang F，Wang Tao. SCB-Dataset3: A Benchmark for Detecting Student Classroom Behavior [J/OL].arXiv:2310.02522 [cs.CV].https://doi.org/10.48550/arXiv.2310.02522.

Funding

Zhengzhou Sias College Campus level Research Project (2025XKE103)

About the authors

Zhang Linjie (1995-), male, Han ethnicity, from Zhoukou, Henan, teacher, master's student, research direction: deep learning, behavior recognition; Ji Zhiyong (1995-), male, Han nationality, from Hebi, Henan, teaching assistant, master's student, research direction: fault diagnosis and prediction, machine learning; Corresponding author: Wang Zhaoxiang (1999-), male, Han nationality, from Xuchang, Henan Province, teaching assistant, master's student, research direction: deep learning, machine vision.