Article Details

Title · Authors · Abstract · Keywords · DOI · References

Review on Low-bit Quantization Methods for Deep Neural Networks

TIAN Cheng, LI Zhengjie, CHEN Gongfu, ZHAO Ming, FENG Shuxuan

(Chengdu Sino-Microelectronics Technology Co., Ltd., Chengdu 610041, China)

Section：Artificial Intelligence · Vol. 2025, Issue 10 · Pages：23-33,38 · 2096-4706（2025）10-0023-12 · CLC number：TP391 · Document code：A · DOI：10.19850/j.cnki.2096-4706.2025.10.006

Abstract

In recent years, Deep Neural Networks (DNNs) have achieved breakthroughs in fields such as Computer Vision and Natural Language Processing. However, DNNs typically contain a large number of parameters and consume substantial computational resources, which impedes their deployment and application on resource-constrained devices. To reduce memory usage and computational burden, the research work on model compression and acceleration has emerged continuously, with model low-bit quantization being one of the primary methods. Model low-bit quantization aims to replace the original high-precision FP32 floating-point operations with more efficient low-precision (≤8 bit) fixed-point or bit operations, thereby significantly reducing the computational cost of the model and enabling the deployment of the quantized network on edge devices. For image classification and object detection tasks in the field of Computer Vision, a comprehensive survey of the current related state of model low-bit quantization techniques is conducted, and the advantages and disadvantages of these methods are analyzed in depth. Additionally, the fundamentals of Deep Neural Networks low-bit quantization are outlined, and the performance of partial representative methods on different datasets is compared. Finally, future development trends in model low-bit quantization are prospected.

Keywords

Deep Neural Networks; model compression and acceleration; low-bit quantization

References

  1. [1] KRIZHEVSKY A，SUTSKEVER I，HINTON G E. Imagenet Classification with Deep Convolutional Neural Networks [J].Advances in Neural Information Processing Systems，2017，60（6）：84-90.
  2. [2] ADARSH P，RATHI P，KUMAR M. YOLO v3-Tiny: Object Detection and Recognition Using One Stage Improved Model [C]//2020 6th International Conference on Advanced Computing and Communication Systems(ICACCS).Coimbatore：IEEE，2020：687-694.
  3. [3] BERTON G，MEREU R，TRIVIGNO G，et al. Deep Visual Geo-Localization Benchmark [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.New Orleans：IEEE，2022：5396-5407.
  4. [4] SAITO K，SOHN K，ZHANG X，et al. Pic2word: Mapping Pictures to Words for Zero-Shot Composed Image Retrieval [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Vancouver：IEEE，2023：19305-19314.
  5. [5] QI D，TAO T，ZHANG Z，et al. AMVP: Adaptive Multi-Volume Primitives for Auto-Driving Novel View Synthesis [J].IEEE Robotics and Automation Letters，2024，9（10）：8306-8313.
  6. [6] SPAGNOLO F，CORSANELLO P，FRUSTACI F，et al. Design of a Low-Power Super-Resolution Architecture for Virtual Reality Wearable Devices [J].IEEE Sensors Journal，2023，23（8）：9009-9016.
  7. [7] KONG Q R，WU Y R，YUAN C，et al. CT-CAD: Context-Aware Transformers for End-to-End Chest Abnormality Detection on X-Rays [C]//IEEE/ACM Transactions on Computational Biology and Bioinformatics.Houston：2023， IEEE，2023：1385-1388.
  8. [8] SIMONYAN K. Very Deep Convolutional Networks for Large-Scale Image Recognition [J/OL].arXiv:1409.1556 [cs.CV].（2014-09-04）.[2024-10-19].https://arxiv.org/abs/1409.1556.
  9. [9] ZHOU A J，MA Y K，ZHU J N，et al. Learning N: M Fine-grained Structured Sparse Neural Networks From Scratch [J/OL].arXiv:2102.04010 [cs.CV].（2021-04-18）.[2024-10-19].https://arxiv.org/abs/2102.04010.
  10. [10] PHAM C，HOANG T，DO T T. Collaborative Multi-Teacher Knowledge Distillation for Learning Low Bit-Width Deep Neural Networks [C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa：IEEE，2023：6435-6443.
  11. [11] SARAGADAM V，BALESTRIERO R，VEERARAGHAVAN A，et al. DeepTensor:Low-Rank Tensor Decomposition with Deep Network Priors [J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2024，46（12）：10337-10348.
  12. [12] XUE Y，CHEN C，SLOWIK A. Neural Architecture Search Based on a Multi-Objective Evolutionary Algorithm with Probability Stack [J].IEEE Transactions on Evolutionary Computation，2023，27（4）：778-786.
  13. [13] JACOB B，KLIGYS S，CHEN B，et al. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City：IEEE，2018：2704-2713.
  14. [14] LIAO Z，QUÉTU V，NGUYEN V T，et al. Can Unstructured Pruning Reduce the Depth in Deep Neural Networks? [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.Paris：IEEE，2023：1402-1406.
  15. [15] REED J K，DEVITO Z，HE H，et al. Torch.fx: Practical Program Capture and Transformation for Deep Learning in Python [J/OL].arXiv:2112.08429 [cs.LG].（2021-12-15） [2022-12-27].https://arxiv.org/abs/2112.08429.
  16. [16] DENG J，DONG W，SOCHER R，et al. Imagenet: A Large-Scale Hierarchical Image Database [C]//2009 IEEE Conference on Computer Vision and Pattern Recognition.Miami：IEEE，2009：248-255.
  17. [17] ZHANG D，YANG J，YE D，et al. LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks [C]//Proceedings of the European Conference on Computer Vision (ECCV).Munich：Springer，2018：365-382.
  18. [18] ZHOU A，YAO A，GUO Y，et al. Incremental Network Quantization：Towards Lossless Cnns with Low-Precision Weights [J/OL].arXiv:1702.03044 [cs.CV].（2017-02-10）[2024-10-19].https://arxiv.org/abs/1702.03044.
  19. [19] ZHOU S，WU Y，NI Z，et al. Dorefa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients [J/OL].arXiv:1606.06160 [cs.NE].（2016-06-20）[2024-10-19].https://arxiv.org/abs/1606.06160.
  20. [20] LI F，LIU B，WANG X，et al. Ternary Weight Networks [C]//ICASSP 2023-2023 IEEE International Conference on Acoustics，Speech and Signal Processing (ICASSP).Rhodes Island：IEEE，2023：1-5.
  21. [21] GONG Y，LIU L，YANG M，et al. Compressing Deep Convolutional Networks Using Vector Quantization [J/OL].arXiv:1412.6115 [cs.CV].（2014-12-10）[2024-10-19].https://arxiv.org/abs/1412.6115.
  22. [22] STOCK P，JOULIN A，GRIBONVAL R，et al. And the Bit Goes Down: Revisiting the Quantization of Neural Networks [J/OL].arXiv:1907.05686 [cs.CV].（2020-11-09）[2024-10-19].https://arxiv.org/abs/1907.05686.
  23. [23] LI Y，DONG X，WANG W. Additive Powers-of-Two Quantization：An Efficient Non-Uniform Discretization for Neural Networks [J/OL].arXiv:1909.13144 [cs.LG].（2019-08-28）[2024-10-19].https://arxiv.org/abs/1909.13144.
  24. [24] FARAO J，FRASER N，BLOTT M，et al. SYQ: Learning Symmetric Quantization for Efficient Deep Neural Networks [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City：IEEE，2018：4300-4309.
  25. [25] SZEGEDY C，LIU W，JIA Y，et al. Going Deeper with Convolutions [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Boston：IEEE，2015：1-9.
  26. [26] HE K，ZHANG X，REN S，et al. Deep Residual Learning for Image Recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas，IEEE：2016：770-778.
  27. [27] IANDOLA F N. SqueezeNet: AlexNet-Level Accuracy with 50x Fewer Parameters and< 0.5 MB Model Size [J/OL].arXiv:1602.07360 [cs.CV].（2016-11-04）[2024-10-19].https://arxiv.org/abs/1602.07360.
  28. [28] LECUN Y，BOTTOU L，BENGIO Y，et al. Gradient-Based Learning Applied to Document Recognition [J].Proceedings of the IEEE，1998，86（11）：2278-2324.
  29. [29] CHEN W，WILSON J，TYREE S，et al. Compressing Neural Networks with the Hashing Trick [C]//International Conference on Machine Learning.Lille：PMLR，2015：2285-2294.
  30. [30] ZHANG A，YANG Z，WANG N，et al. Comq:A Backpropagation-Free Algorithm for Post-Training Quantization [J/OL].arXiv:2403.07134 [cs.LG].（2024-03-11）[2024-10-19].https://arxiv.org/abs/2403.07134.
  31. [31] LI X，LIU B，YANG R H，et al. DenseShift: Towards Accurate and Efficient Low-Bit Power-of-Two Quantization [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.Las Vegas：IEEE，2023：17010-17020.
  32. [32] LI J，CHEN G，JIN M，et al. AE-Qdrop: Towards Accurate and Efficient Low-Bit Post-Training Quantization for a Convolutional Neural Network [J].Electronics，2024，13（3）：644.
  33. [33] SANDLER M，HOWARD A，ZHU M，et al. Mobilenetv2: Inverted Residuals and Linear Bottlenecks [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City：IEEE，2018：4510-4520.
  34. [34] TAN M，LE Q. Efficientnet: Rethinking Model Scaling for Convolutional Neural Networks [C]//International Conference on Machine Learning.PMLR，2019：6105-6114.
  35. [35] BANNER R，NAHSHAN Y，SOUDRY D. Post Training 4-Bit Quantization of Convolutional Networks for Rapid-Deployment [J/OL].arXiv:1810.05723 [cs.CV].（2018-10-02）[2024-10-19].https://arxiv.org/abs/1810.05723.
  36. [36] LOU Q，GUO F，LIU L，et al. AutoQ: Automated Kernel-Wise Neural Network Quantization [J].arXiv:1902.05690 [cs.LG].（2019-02-15）[2024-10-19].https://arxiv.org/abs/1902.05690.
  37. [37] XU K，LI Z，WANG S，et al. PTMQ: Post-Training Multi-Bit Quantization of Neural Networks [C]//Proceedings of the AAAI Conference on Artificial Intelligence.Washington：AAAI Press，2024：16193-16201.
  38. [38] YANG W，ZHI X，TONG W. Optimization of Linear Quantization for General and Effective Low Bit-Width Network Compression [J/OL].Algorithms，2023，16（1）：31（2023-01-04）[2024-10-19].https://doi.org/10.3390/a16010031.
  39. [39] CHAUHAN A，TIWARI U. Post Training Mixed Precision Quantization of Neural Networks Using First-Order Information [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Paris：IEEE，2023：1343-1352.
  40. [40] IANDOLA F N，HAN S，MOSKEWICZ M W，et al. SqueezeNet: Alexnet-Level Accuracy with 50x Fewer Parameters and<0.5 MB Model Size [J/OL].arXiv:1602.07360 [cs.CV].（2016-02-24）[2024-10-19].https://arxiv.org/abs/1602.07360.
  41. [41] SZEGEDY C，VANHOUCKE V，IOFFE S，et al. Rethinking the Inception Architecture for Computer Vision [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas：IEEE，2016：2818-2826.
  42. [42] XU K，HAN L，TIAN Y，et al. Eq-net: Elastic Quantization Neural Networks [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.Paris：IEEE，2023：1505-1514.
  43. [43] BENGIO Y，LÉONARD N，COURVILLE A. Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation [J/OL].arXiv:1308.3432 [cs.LG].（2013-08-15）[2024-10-19].https://arxiv.org/abs/1308.3432.
  44. [44] XU Q，CHEN Z，RAGAB M，et al. Contrastive Adversarial Knowledge Distillation for Deep Model Compression in Time-Series Regression Tasks [J].Neurocomputing，2022，485：242-251.
  45. [45] QU Z，ZHOU Z，CHENG Y，et al. Adaptive Loss-Aware Quantization for Multi-Bit Networks [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Seattle：IEEE，2020：7988-7997.
  46. [46] YANG L，JIN Q. Fracbits: Mixed Precision Quantization via Fractional Bit-Widths [C]//Proceedings of the AAAI Conference on Artificial Intelligence.AAAI Press，2021，35（12）：10612-10620.
  47. [47] ZHAO S，YUE T，HU X. Distribution-Aware Adaptive Multi-Bit Quantization [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville：IEEE，2021：9281-9290.
  48. [48] SHER A，TRUSOV A，LIMONOVA E，et al. Neuron-by-Neuron Quantization for Efficient Low-Bit QNN Training [J].Mathematics，2023，11（9）：2112.
  49. [49] LUO R，DRIMBAREAN A，MCDERMOTT J，et al. Reclaiming Residual Knowledge: A Novel Paradigm to Low-Bit Quantization [J/OL].arXiv:2408.00923 [cs.CV].（2024-08-01）[2024-10-19].https://arxiv.org/abs/2408.00923.
  50. [50] ELTHAKEB A T，PILLIGUNDLA P，MIRESHGHALLAGH F S，et al. ReLeQ: A Reinforcement Learning Approach for Deep Quantization of Neural Networks [J].IEEE Micro，2020，40（5）：37-45.
  51. [51] WANG K，LIU Z，LIN Y，et al. HAQ: Hardware-Aware Automated Quantization with Mixed Precision [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Long Beach：IEEE，2019：8612-8620.
  52. [52] WU B，WANG Y，ZHANG P，et al. Mixed Precision Quantization of Convnets via Differentiable Neural Architecture Search [J/OL].arXiv:1812.00090 [cs.CV].（2018-11-30）[2024-10-19].https://arxiv.org/abs/1812.00090.
  53. [53] DONG Z，YAO Z，GHOLAMI A，et al. HAWQ: Hessian Aware Quantization of Neural Networks with Mixed-Precision [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.Seoul：IEEE，2019：293-302.
  54. [54] LIU W，ANGUELOV D，ERHAN D，et al. SSD: Single Shot Multibox Detector [C]//Computer Vision–ECCV 2016.Amsterdam:Springer，2016：21-37.
  55. [55] REDMON J. You Only Look Once: Unified, Real-Time Object Detection [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas：IEEE，2016：779-788.
  56. [56] HE K，GKOIXARI G，DOLLÁR P，et al. Mask R-CNN [C]//Proceedings of the IEEE International Conference on Computer Vision.Venice：IEEE，2017：2961-2969.
  57. [57] LIN T Y，MAIRE M，BELONGIE S，et al. Microsoft COCO: Common Objects in Context[C]//Computer Vision-ECCV 2014.Zurich：Springer International Publishing，2014：740-755.
  58. [58] GUPTA K，ASTHANA A. Reducing the Side-Effects of Oscillations in Training of Quantized YOLO Networks [C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.Waikoloa：IEEE，2024：2452-2461.
  59. [59] ESSER S K，MCKINSTRY J L，BABLANI D，et al. Learned Step Size Quantization [J/OL].arXiv:1902.08153 [cs.LG].（2019-02-21）[2024-10-19].https://arxiv.org/abs/1902.08153.
  60. [60] LI R，WANG Y，LIANG F，et al. Fully Quantized Network for Object Detection [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach：IEEE，2019：2805-2814.
  61. [61] CHEN P，LIU J，ZHUANG B，et al. AQD: Towards Accurate Quantized Object Detection [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Nashville：IEEE，2021：104-113.
  62. [62] ZHUANG B，LIU L，TAN M，et al. Training Quantized Neural Networks with a Full-Precision Auxiliary Module [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle：IEEE，2020：1488-1497.
  63. [63] BUCHNEV V，HE J，SUN F，et al. RUPQ: Improving Low-Bit Quantization by Equalizing Relative Updates of Quantization Parameters [C]//The 34th British Machine Vision Conference.Aberdeen：BMVA，2023：307-309.
  64. [64] ULTRALYTICS J. YOLOv5: A State-of-the-Art Real-Time Object Detection System [J/OL].[2024-10-19].https://iq.opengenus.org/yolov5/.
  65. [65] WANG C Y，BOCHKOVSKIY A，LIAO H Y M. YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.Vancouver：IEEE，2023：7464-7475.
  66. [66] LIN T Y，GOYAL P，GIRSHICK R，et al. Focal Loss for Dense Object Detection [J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，42（2）：318-327.
  67. [67] SHI X，DING J，HAO Z，et al. Towards Energy Efficient Spiking Neural Networks: An Unstructured Pruning Framework [C]//The Twelfth International Conference on Learning Representations.Vienna：ICLR，2024：1-21.
  68. [68] GUO H，GREENGARD P，XING E P，et al. LQ-LoRa: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning [J/OL].arXiv:2311.12023 [cs.CL].（2023-11-20）[2024-10-19].https://arxiv.org/abs/2311.12023.
  69. [69] ELSHAWI R，MAHER M，SAKR S. Automated Machine Learning: State-of-the-Art and Open Challenges [J/OL].arXiv:1906.02287 [cs.LG].（2019-06-05）[2024-10-19].https://arxiv.org/abs/1906.02287.

About the authors

Tian Cheng (1998-), female, Han ethnicity, from Bijie, Guizhou, is a visual algorithm FPGA acceleration engineer with a master's degree. Her research direction is model quantification and acceleration; Li Zhengjie (1983-), male, Han ethnicity, born in Chengdu, Sichuan, is a senior engineer with a PhD. His research direction is FPGA acceleration for visual algorithms; Chen Gongfu (1996-), male, Han nationality, from Xinyang, Henan, engineer, bachelor's degree, research direction: NPU design; Zhao Ming (1987-), male, Han ethnicity, from Shehong, Sichuan, engineer, master's degree, research direction: FPGA acceleration of AI algorithms; Feng Shuxuan (2001-), male, Han ethnicity, from Haikou, Hainan, is currently pursuing a master's degree in AI chip design.