Article Details

Title · Authors · Abstract · Keywords · DOI · References

Survey of Vehicle 3D Object Detection Methods Based on Monocular Scene

TANG Xinyao1,2, WANG Wei3

(1.School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510640, China; 2.GRG Banking Equipment Co., Ltd., Guangzhou 510663, China; 3.School of Information Engineering, Chang'an University, Xi'an 710018, China)

Section：Artificial Intelligence · Vol. 2025, Issue 9 · Pages：16-24,31 · 2096-4706（2025）09-0016-10 · CLC number：TP391.4 · Document code：A · DOI：10.19850/j.cnki.2096-4706.2025.09.004

Abstract

In recent years, vehicle 3D Object Detection has attracted much attention in the field of intelligent transportation such as autonomous driving. Compared with 2D detection, 3D detection can accurately estimate the position, size and attitude of the target in 3D space. Because of the advantages of low cost and high data processing efficiency, monocular camera plays a leading role in practical applications. This paper focuses on the 3D Object Detection method in monocular scenes and systematically sorts out its development context. Firstly, according to the source of prior information, the detection methods are divided into three categories based on geometric information, 2D Object Detection and geometric information constraints, and 3D feature estimation. The core ideas, advantages and disadvantages of representative algorithms in each category are analyzed. Secondly, the common data sets and evaluation indexes commonly used in the field are introduced, and the experimental results of typical algorithms are quantitatively compared on the KITTI dataset. Finally, combined with the current research status, the main existing problems in this field are analyzed, and the future development trend is prospected.

Keywords

intelligent transportation; monocular camera; 3D Object Detection; Deep Learning

References

  1. [1] ASEEM B，OMID H J，SIVA K M，et al. Bounding Boxes, Segmentations and Object Coordinates: How Important is Recognition for 3D Scene Flow Estimation in Autonomous Driving Scenarios? [C]//2017 IEEE International Conference on Computer Vision(ICCV).Venice：IEEE，2017：2593-2602.
  2. [2] MA X Z，WANG Z H，LI H J，et al. Accurate Monocular 3D Object Detection Via Color-Embedded 3D Reconstruction for Autonomous Driving [C]//2019 IEEE /CVF International Conference on Computer Vision(ICCV).Seoul：IEEE，2019：6850-6859.
  3. [3] Cao Pei Dual Sensor Information Fusion Target Detection and Attitude Estimation for Autonomous Driving [D]. Harbin: Harbin Institute of Technology, 2019
  4. [4] ZHANG H R，CAO Q X. Texture-Less Object Detection and 6D Pose Estimation in RGB-D Images [J].Robotics and Autonomous Systems，2017，95：64-79.
  5. [5] DAVID J T，NASSIR N，FEDERICO T. 6D Object Pose Estimation with Depth Images: A Seamless Approach for Robotic Interaction and Augmented Reality [J/OL].arXiv:1709.01459 [cs.CV].[2024-12-18].https://arxiv.org/abs/1709.01459.
  6. [6] JORGE B，CARLOS G，FRANCISCO M M，et al. Birdnet: A 3D Object Detection Framework from LiDAR Information [C]//2018 21st International Conference on Intelligent Transportation Systems(ITSC).Maui：IEEE，2018：3517-3523.
  7. [7] LI B，ZHANG T L，XIA T. Vehicle Detection from 3D Lidar Using Fully Convolutional Network [J/OL].arXiv:1608.07916 [cs.CV].[2024-12-19].https://arxiv.org/abs/1608.07916.
  8. [8] SHI S S，WANG X G，LI H S. Pointrcnn: 3D Object Proposal Generation and Detection from Point Cloud [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Long Beach：IEEE，2019：770-779.
  9. [9] ZHOU Y，ONCEL T. Stereo Imagery: End-to-end Learning for Point Cloud Based 3D Object Detection [C]//2018 IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City：IEEE，2018：4490-4499.
  10. [10] GUPTA S，GIRSHICK R，ARBELÁEZ P，et al. Learning Rich Features from RGB-D Images for Object Detection and Segmentation [J].CoRR，2014：345-360.
  11. [11] CHEN X Z，KAUSTAV K，ZHU Y K，et al. 3D Object Proposals Using Stereo Imagery for Accurate Object Class Detection [J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2018，40（5）：1259-1272.
  12. [12] XU B，CHEN Z Z. Multi-Level Fusion Based 3D Object Detection from Monocular Images [C]//2018 IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City：IEEE，2018：2345-2353.
  13. [13] QI C R，YI L，SU H，et al. PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space [J/OL].arXiv:1706.02413 [cs.CV].[2024-12-19].https://arxiv.org/abs/1706.02413.
  14. [14] CHEN X Z，MA H M，WAN J，et al. Multi-View 3D Object Detection Network for Autonomous Driving [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu：IEEE，2017：6526-6534.
  15. [15] QI C R，LIU W，WU C X，et al. Frustum PointNets for 3D Object Detection from RGB-D Data [C]//2018 IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City：IEEE，2018：918-927.
  16. [16] CHU H，MA W C，KAUSTAV K，et al. Surfconv: Bridging 3D and 2D Convolution for RGBD Images [C]//2018 IEEE Conference on Computer Vision and Pattern Recognition.Salt Lake City：IEEE，2018：3002-3011.
  17. [17] QIN Z Y，WANG J L，LU Y. Triangulation Learning Network: From Monocular to Stereo 3D Object Detection [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach：IEEE，2019：7607-7615.
  18. [18] LI P L，CHEN X Z，SHEN S J. Stereo R-CNN Based 3D Object Detection for Autonomous Driving [C]//2019 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Long Beach：IEEE，2019：7636-7644.
  19. [19] HU H N，CAI Q Z，WANG D Q，et al. Joint Monocular 3D Vehicle Detection and Tracking [C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV).Seoul：IEEE，2019：5389-5398.
  20. [20] QIN Z Y，WANG J L，LU Y. Monogrnet: A Geometric Reasoning Network for Monocular 3D Object Localization [J].Proceedings of the AAAI Conference on Artificial Intelligence，2019，33：8851-8858.
  21. [21] THOMAS R，ALEX K，ROBERTO C. Orthographic Feature Transform for Monocular 3D Object Detection [J/OL].arXiv:1811.08188 [cs.CV].[2024-12-19].https://arxiv.org/abs/1811.08188.
  22. [22] KU J，PON A D，WASLANDER S L. Monocular 3D Object Detection Leveraging Accurate Proposals and Shape Reconstruction [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Long Beach：IEEE，2019：11859-11868.
  23. [23] XIANG Y，CHOI W G，LIN Y Q，et al. Subcategory Aware Convolutional Neural Networks for Object Proposals and Detection [C]//2017 IEEE Winter Conference on Applications of Computer Vision (WACV).Santa Rosa：IEEE，2017：924-933.
  24. [24] LI C，ZIA M Z，TRAN Q H，et al. Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsing [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu：IEEE，2017：388-397.
  25. [25] ZIA M Z，MICHAEL S，BERNT S，et al. Detailed 3D Representations for Object Recognition and Modeling [J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2013，35（11）：2608-2623.
  26. [26] DUBSKÁ M，HEROUT A，SOCHOR J. Automatic Camera Calibration for Traffic Understanding [EB/OL].[2024-12-19].https://www.bmva-archive.org.uk/bmvc/2014/papers/paper013/index.html.
  27. [27] ultralytics. YOLOv5 [EB/OL].[2024-12-19].https://github.com/ultralytics/yolov5.
  28. [28] ZHOU X Y，WANG D Q，KRHENBHL P. Objects as Points [J/OL].arXiv:1904.07850 [cs.CV].[2024-12-20].https://arxiv.org/abs/1904.07850.
  29. [29] LIU W，ANGUELOV D，ERHAN D，et al. SSD: Single Shot MultiBox Detector [J/OL].arXiv:1512.02325 [cs.CV].[2024-12-20].https://arxiv.org/abs/1512.02325.
  30. [30] LIU L J，LU J W，XU C J，et al. Deep Fitting Degree Scoring Network for Monocular 3D Object Detection [C]//2019 IEEE Conference on Computer Vision and Pattern Recognition.Long Beach：IEEE，2019：1057-1066.
  31. [31] CHEN X Z，KAUSTAV K，ZHANG Z Y，et al. Monocular 3D Object Detection for Autonomous Driving [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Las Vegas：IEEE，2016：2147-2156.
  32. [32] MOUSAVIAN A，ANGUELOV D，FLYNN J，et al. 3D Bounding Box Estimation Using Deep Learning and Geometry [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.Honolulu：IEEE，2017：5632-5640.
  33. [33] CHABOT F，CHAOUCH M，RABARISOA J，et al. Deep MANTA: A Coarse-to-Fine Many-Task Network for Joint 2D and 3D Vehicle Analysis from Monocular Image [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).Honolulu：IEEE，2017：1827-1836.
  34. [34] YANG S C，SCHERER S. CubeSLAM: Monocular 3-D Object SLAM [J].IEEE Transactions on Robotics，2019，35（4）：925-938.
  35. [35] HE T，SOATTO S. Mono3D++: Monocular 3D Vehicle Detection with Two-Scale 3D Hypotheses and Task Priors [J/OL].arXiv:1901.03446 [cs.CV].[2024-12-19].https://arxiv.org/abs/1901.03446.
  36. [36] MANHARDT F，KEHL W，GAIDON A. ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shape [C]//2019 IEEE Conference on Computer Vision and Pattern Recognition（CVPR）.Long Beach：IEEE，2019：2064-2073.
  37. [37] DING M Y，HUO Y Q，YI H W，et al. Learning Depth-Guided Convolutions for Monocular 3D Object Detection [C]//2020 IEEE Conference on Computer Vision and Pattern Recognition.Seattle：IEEE，2020：11669-11678.
  38. [38] M RAD，VINCENT L. BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth [C]//2017 IEEE International Conference on Computer Vision.Venice：IEEE，2017：3848-3856.
  39. [39] WADIM K，FABIAN M，FEDERICO T，et al. SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again [C]//2017 IEEE International Conference on Computer Vision.Venice：IEEE，2017：1530-1538.
  40. [40] BUGRA T，SUDIPTA N S，PASCAL F. Real-Time Seamless Single Shot 6D Object Pose Prediction [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City：IEEE，2018：292-301.
  41. [41] ISHAN G，AKSHAY R，MOHAN T. 3D Bounding Boxes for Road Vehicles: A One-Stage, Localization Prioritized Approach Using Single Monocular Images [EB/OL].[2024-12-19]. https://openaccess.thecvf.com/content_eccv_2018_workshops/w29/html/Gupta_3D_Bounding_Boxes_for_Road_Vehicles_A_One-Stage_Localization_Prioritized_ECCVW_2018_paper.html.
  42. [42] LI B Y，OUYANG W L，SHENG L，et al. GS3D: An EFFicient 3D Object Detection Framework for Autonomous Driving [C]//2019 IEEE Conference on Computer Vision and Pattern Recognition.Long Beach：IEEE，2019：1019-1028.
  43. [43] LI P X，ZHAO H C，LIU P F，et al. RTM3D： Real-Time Monocular 3D Detection from Object Keypoints for Autonomous Driving [J/OL].arXiv:2001.03343 [cs.CV].[2024-12-19].https://arxiv.org/abs/2001.03343v1.
  44. [44] IVAN B，ALEXEY A，EVGENY B，et al. Monocular 3D Object Detection via Geometric Reasoning on Keypoints [J/OL].arXiv:1905.05618 [cs.CV].[2024-12-19].https://arxiv.org/abs/1905.05618.
  45. [45] ANDREAS G，PHILIP L，RAQUEL U. Are we Ready for Autonomous Driving? The KITTI Vision Benchmark Suite [C]//2012 IEEE Conference on Computer Vision and Pattern Recognition.Providence：IEEE，2012：3354-3361.
  46. [46] XIANG Y，ROOZBEH M，SILVIO S. Beyond PASCAL: A Benchmark for 3D Object Detection in the Wild [C]//IEEE Winter Conference on Applications of Computer Vision.Steamboat Springs：IEEE，2014：75-82.
  47. [47] HUANG X Y，WANG P，CHENG X J，et al. The ApolloScape Open Dataset for Autonomous Driving and Its Application [J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2018，42（10）：2702-2719.
  48. [48] SUN P，HENRIK K，XERXES D，et al. Scalability in Perception for Autonomous Driving: Waymo Open Dataset [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Seattle：IEEE，2020：2443-2451.
  49. [49] CAESAR H，BANKITI V，LANG A H，et al. NuScenes: A Multimodal Dataset for Autonomous Driving [C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Seattle：IEEE，2020：11618-11628.

Funding

Shaanxi Provincial Natural Science Basic Research Program Project (2023-JC-YB-600)

About the authors

Tang Xinyao (1996-), female, Han ethnicity, from Suzhou, Jiangsu, is an algorithm engineer with a PhD. Her research interests include computer vision and intelligent transportation.