Computer Vision-Based Intelligent Traffic Surveillance: Multi-Vehicle Tracking and Detection


  • Amir Mahmud Husein Universitas Prima Indonesia
  • Kevi Noflianhar Lubis Universitas Prima Indonesia
  • Daniel Salim Sidabutar Universitas Prima Indonesia
  • Yansan Yuanda Universitas Prima Indonesia
  • Kevry Universitas Prima Indonesia
  • Ashwini Waren Universitas Prima Indonesia, Indonesia




Computer Vision; Deep Learning; Traffic Surveillance; Vehicle Detection; YOLOv4


The application of vehicle detection in real-time traffic surveillance systems is one of the challenging research fields with different objectives. One of the problems is the detection of many vehicles simultaneously in a video sequence sourced from CCTV cameras. In many works, the focus is only on detecting vehicle classes such as motorcycles, buses, trucks, and cars or special vehicles such as ambulances and others. In this research, we propose to apply 13 classes of vehicle types and implement YOLOv4 in the traffic surveillance task. More specifically, all classes are labeled, and then the YOLOv4 model is trained on 800 images and tested on 23 videos from three intersections in Medan City, namely Juanda Katamso Intersection, Gatot Subroto Intersection, and Uniland Intersection. Based on the test results, YOLOv4 proves successful in detecting many vehicles in frame-by-frame sequence with various types of vehicles. All vehicle detection data will be stored in the file.

GS Cited Analysis


Download data is not yet available.


Ammar, A., Koubaa, A., Ahmed, M., Saad, A., & Benjdira, B. (2021). Vehicle detection from aerial images using deep learning: A comparative study. Electronics (Switzerland), 10(7), 1–31.

Bochkovskiy, A., Wang, C.-Y., & Liao, H.-Y. M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. ArXiv. Retrieved from

Butt, M. A., Khattak, A. M., Shafique, S., Hayat, B., Abid, S., Kim, K. Il, … Adnan, A. (2021). Convolutional Neural Network Based Vehicle Classification in Adverse Illuminous Conditions for Intelligent Transportation Systems. Complexity, 2021.

Choi, J., Chun, D., Kim, H., & Lee, H. J. (2019). Gaussian YOLOv3: An accurate and fast object detector using localization uncertainty for autonomous driving. Proceedings of the IEEE International Conference on Computer Vision, 2019-Octob, 502–511.

Du, S., Zhang, P., Zhang, B., & Xu, H. (2021). Weak and Occluded Vehicle Detection in Complex Infrared Environment Based on Improved YOLOv4. IEEE Access, 9, 25671–25680.

Han, B. G., Lee, J. G., Lim, K. T., & Choi, D. H. (2020). Design of a scalable and fast yolo for edge-computing devices. Sensors (Switzerland), 20(23), 1–15.

Huang, Y. Q., Zheng, J. C., Sun, S. D., Yang, C. F., & Liu, J. (2020). Optimized YOLOv3 algorithm and its application in traffic flow detections. Applied Sciences (Switzerland), 10(9), 3079.

Husein, A. M., Christopher, C., Gracia, A., Brandlee, R., & Hasibuan, M. H. (2020). Deep Neural Networks Approach for Monitoring Vehicles on the Highway. SinkrOn, 4(2), 163.

Kasper-Eulaers, M., Hahn, N., Kummervold, P. E., Berger, S., Sebulonsen, T., & Myrland, Ø. (2021). Short communication: Detecting heavy goods vehicles in rest areas in winter conditions using YOLOv5. Algorithms, 14(4).

Li, X., Liu, Y., Zhao, Z., Zhang, Y., & He, L. (2018). A deep learning approach of vehicle multitarget detection from traffic video. Journal of Advanced Transportation, 2018.

Liang, R., & Ji, G. (2022). Vehicle Detection Algorithm Based on Embedded Video Image Processing in the Background of Information Technology. Journal of Electrical and Computer Engineering, 2022.

Liu, C., Zhang, Y., Luo, H., Tang, J., Chen, W., Xu, X., … Shen, Y. D. (2021). City-scale multi-camera vehicle tracking guided by crossroad zones. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, (1), 4124–4132.

Liu, Y., & Zhang, G. (2021). Vehicle detection algorithm based on LW-SSD. Journal of Physics: Conference Series, 1748(3).

Lu, J., Xia, M., Gao, X., Yang, X., Tao, T., Meng, H., … DIng, E. (2021). Robust and online vehicle counting at crowded intersections. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 3997–4003.

Luo, J., Fang, H., Shao, F., Zhong, Y., & Hua, X. (2021). Multi-scale traffic vehicle detection based on faster R–CNN with NAS optimization and feature enrichment. Defence Technology, 17(4), 1542–1554.

Mahto, P., Garg, P., Seth, P., & Panda, J. (2020). Refining Yolov4 for vehicle detection. International Journal of Advanced Research in Engineering and Technology, 11(5), 409–419.

Meng, Q., Song, H., Zhang, Y., Zhang, X., Li, G., & Yang, Y. (2020). Video-Based Vehicle Counting for Expressway: A Novel Approach Based on Vehicle Detection and Correlation-Matched Tracking Using Image Data from PTZ Cameras. Mathematical Problems in Engineering, 2020.

Nam Bui, K. H., Yi, H., & Cho, J. (2020). A multi-class multi-movement vehicle counting framework for traffic analysis in complex areas using CCTV systems. Energies, 13(8).

Neupane, B., Horanont, T., & Aryal, J. (2022). Real-Time Vehicle Classification and Tracking Using a Transfer Learning-Improved Deep Learning Network. Sensors, 22(10), 1–21.

Sun, L., Zhang, J., Yang, Z., & Fan, B. (2023). A Motion-Aware Siamese Framework for Unmanned Aerial Vehicle Tracking. Drones, 7(3), 1–24.

Tian, E., & Kim, J. (2023). Improved Vehicle Detection Using Weather Classification and Faster R-CNN with Dark Channel Prior. Electronics (Switzerland), 12(14).

Wang, X., Wang, S., Cao, J., & Wang, Y. (2020). Data-Driven Based Tiny-YOLOv3 Method for Front Vehicle Detection Inducing SPP-Net. IEEE Access, 8, 110227–110236.

Yang, H., Zhang, Y., Zhang, Y., Meng, H., Li, S., & Dai, X. (2021). A Fast Vehicle Counting and Traffic Volume Estimation Method Based on Convolutional Neural Network. IEEE Access, 9, 150522–150531.

Yu, B., Shin, J., Kim, G., Roh, S., & Sohn, K. (2021). Non-Anchor-Based Vehicle Detection for Traffic Surveillance Using Bounding Ellipses. IEEE Access, 9, 123061–123074.

Zhao, J., Hao, S., Dai, C., Zhang, H., Zhao, L., Ji, Z., & Ganchev, I. (2022). Improved Vision-Based Vehicle Detection and Classification by Optimized YOLOv4. IEEE Access, 10(i), 8590–8603.

Zhao, M., Zhong, Y., Sun, D., & Chen, Y. (2021). Accurate and efficient vehicle detection framework based on SSD algorithm. IET Image Processing, 15(13), 3094–3104.

Zhu, J., Li, X., Jin, P., Xu, Q., Sun, Z., & Song, X. (2021). MME-YOLO: Multi-sensor multi-level enhanced yolo for robust vehicle detection in traffic surveillance. Sensors (Switzerland), 21(1), 1–17.


Crossmark Updates

How to Cite

Husein, A. M., Noflianhar Lubis, K., Salim Sidabutar, D., Yuanda, Y., Kevry, & Waren, A. . (2024). Computer Vision-Based Intelligent Traffic Surveillance: Multi-Vehicle Tracking and Detection. Sinkron : Jurnal Dan Penelitian Teknik Informatika, 9(1), 384-391.

Most read articles by the same author(s)

1 2 > >>