Real-Time Ego-Lane Detection in a Low-Cost Embedded Platform using CUDA-Based Implementation
DOI:
https://doi.org/10.5433/1679-0375.2023.v44.48268Keywords:
ego-lane detection, real-time application, CUDA, heterogeneous computingAbstract
This work assesses the effectiveness of heterogeneous computing based on a CUDA implementation for real-time ego-lane detection using a typical low-cost embedded computer. We propose and evaluate a CUDA-optimized algorithm using a heterogeneous approach based on the extraction of features from an aerial perspective image. The method incorporates well-known algorithms optimized to achieve a very efficient solution with high detection rates and combines techniques to enhance markings and remove noise. The CUDA-based solution is compared to an OpenCV library and to a serial CPU implementation. Practical experiments using TuSimple's image datasets were conducted in an NVIDIA's Jetson Nano embedded computer. The algorithm detects up to 97.9% of the ego lanes with an accuracy of 99.0% in the best-evaluated scenario. Furthermore, the CUDA-optimized method performs at rates greater than 300 fps in the Jetson Nano embedded system, speeding up 25 and 140 times the OpenCV and CPU implementations at the same platform, respectively. These results show that more complex algorithms and solutions can be employed for better detection rates while maintaining real-time requirements in a typical low-power embedded computer using a CUDA implementation.
Metrics
References
Afif, M., Said, Y., & Atri, M. (2020). Computer vision algorithms acceleration using graphic processors NVIDIA CUDA. Cluster Computing, 23(4), 3335–3347. DOI: https://doi.org/10.1007/s10586-020-03090-6
Borkar, A., Hayes, M., & Smith, M. T. (2012). A novel lane detection system with efficient ground truth generation. IEEE Transactions on Intelligent Transportation Systems, 13(1), 365–374. DOI: https://doi.org/10.1109/TITS.2011.2173196
Borkar, A., Hayes, M., & Smith, M. T. (2009). Robust lane detection and tracking with Ransac and Kalman filter. In Institute of Electrical and Electronics Engineers, Conferences [Proceedings]. 16th IEEE International Conference on Image Processing (ICIP), Cairo, Egypt. DOI: https://doi.org/10.1109/ICIP.2009.5413980
Bradski, G. (2000). The OpenCV Library. Dr. Dobb’s Journal of Software Tools, 120, 122–125.
Cao, J., Song, S., Xiao, W., & Peng, Z. (2019). Lane detection algorithm for intelligent vehicles in complex road conditions and dynamic environments. Sensors, 19, 3166. DOI: https://doi.org/10.3390/s19143166
Gansbeke, W. V., Brabandere, B. D., Neven, D., Proesmans, M., & Gool, L. V. (2019). End-to-end Lane Detection through Differentiable Least-Squares Fitting. ArXiv:1902.00293v3 [cs.CV].
Hartley, R., & Zisserman, A. (2003). Multiple View Geometry in Computer Vision (2nd ed.). Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511811685
He, B., Ai, R., Yan, Y., & Lang, X. (2016). Accurate and robust lane detection based on Dual-View Convolutional Neutral Network. In Institute of Electrical and Electronics Engineers, Conferences [Proceedings]. IEEE 4th Intelligent Vehicles Symposium, Gothenburg, Sweden.
Hernández, D. C., Filonenko, A., Shahbaz, A., & Jo, K.-H. (2017). Lane marking detection using image features and line fitting model. In Institute of Electrical and Electronics Engineers, Conferences [Proceedings]. 10th International Conference on Human System Interactions (HSI), Ulsan, Korea.
Hillel, A. B., Lerner, R., Levi, D., & Raz, G. (2014). Recent progress in road and lane detection: A survey. Machine Vision and Applications, 25, 727–74. DOI: https://doi.org/10.1007/s00138-011-0404-2
Huang, Y., Li, Y., Hu, X., & Ci, W. (2018). Lane Detection Based on Inverse Perspective Transformation and Kalman Filter. KSII Transactions on Internet and Information Systems. Korean Society for Internet Information (KSII), 12(2), 643–661. DOI: https://doi.org/10.3837/tiis.2018.02.006
Jaiswal, D., & Kumar, P. (2020). Real-time implementation of moving object detection in UAV videos using GPUs. Journal of Real-Time Image Processing, 17(5), 1301–1317. DOI: https://doi.org/10.1007/s11554-019-00888-5
Kim, H.-S., Beak, S.-H., & Park, S.-Y. (2016). Parallel Hough Space Image Generation Method for RealTime Lane Detection. In J. Blanc-Talon, C. Distante, W. Philips, D. Popescu, & P. Scheunders (Eds.), Advanced Concepts for Intelligent Vision Systems (pp. 81–91). Springer International Publishing. DOI: https://doi.org/10.1007/978-3-319-48680-2_8
Kim, J., Kim, J., Jang, G.-J., & Lee, M. (2017). Fast learning method for convolutional neural networks using extreme learning machine and its application to lane detection. Neural Networks, 87, 109–121. DOI: https://doi.org/10.1016/j.neunet.2016.12.002
Kirk, D. B., & Hwu, W.-M. W. (2016). Programming massively parallel processors: A hands-on approach (3 th ed.). Elsevier.
Küçükmanisa, A., Tarim, G., & Urhan, O. (2019). Realtime illumination and shadow invariant lane detection on mobile platform. Journal of Real-Time Image Processing, 16(5), 1781–1794. DOI: https://doi.org/10.1007/s11554-017-0687-2
Kühnl, T., Kummert, F., & Fritsch, J. (2012). Spatial ray features for real-time ego-lane extraction. In Institute of Electrical and Electronics Engineers, Conferences [Proceedings]. 15th International IEEE Conference on Intelligent Transportation Systems, Anchorage, USA. DOI: https://doi.org/10.1109/ITSC.2012.6338740
Lee, C., & Moon, J.-H. (2018). Robust Lane Detection and Tracking for Real-Time Applications. IEEE Transactions on Intelligent Transportation Systems, 19(12), 4043–4048. DOI: https://doi.org/10.1109/TITS.2018.2791572
Li, J., Deng, G., Zhang, W., Zhang, C., Wang, F., & Liu, Y. (2020). Realization of CUDA-based real-time multi-camera visual SLAM in embedded systems. Journal of Real-Time Image Processing, 17(3), 713–727. DOI: https://doi.org/10.1007/s11554-019-00924-4
Li, W., Qu, F., Wang, Y., Wang, L., & Chen, Y. (2019). A robust lane detection method based on hyperbolic model. Soft Computing, 23(19), 9161–9174. DOI: https://doi.org/10.1007/s00500-018-3607-x
Li, X., Fang, X., Ci, W., & Zhang, W. (2014). Lane Detection and Tracking Using a Parallel-snake Approach. Journal of Intelligent and Robotic Systems, 77(3), 597–609. DOI: https://doi.org/10.1007/s10846-014-0075-0
Mammeri, A., Boukerche, A., & Tang, Z. (2016). A realtime lane marking localization, tracking and communication system. Computer Communications, 73, 132–143. DOI: https://doi.org/10.1016/j.comcom.2015.08.010
Muthalagu, R., Bolimera, A., & Kalaichelvi, V. (2020). Lane detection technique based on perspective transformation and histogram analysis for selfdriving cars. Computers & Electrical Engineering, 85, 106653. DOI: https://doi.org/10.1016/j.compeleceng.2020.106653
Narote, S. P., Bhujbal, P. N., Narote, A. S., & Dhane, D. M. (2018). A review of recent advances in lane detection and departure warning system. Pattern Recognition, 73, 216–234. DOI: https://doi.org/10.1016/j.patcog.2017.08.014
Nguyen, V., Kim, H., Jun, S., & Boo, K. (2018). A study on real-time detection method of lane and vehicle for lane change assistant system using vision system on highway. Engineering Science and Technology, an International Journal, 21(5), 822–833. DOI: https://doi.org/10.1016/j.jestch.2018.06.006
Nvidia. (2011). NVIDIA CUDA Programming Guide. NVIDIA Corporation.
Nvidia. (2014). GeForce GTX 980 - Featuring Maxwell, The Most Advanced GPU Ever Made. NVIDIA Corporation.
Paula, M. B., & Jung, C. R. (2015). Automatic Detection and Classification of Road Lane Markings Using Onboard Vehicular Cameras. IEEE Transactions on Intelligent Transportation Systems, 16(6), 3160–3169. DOI: https://doi.org/10.1109/TITS.2015.2438714
Philion, J. (2019). FastDraw: Addressing the Long Tail of Lane Detection by Adapting a Sequential Prediction Network. In Institute of Electrical and Electronics Engineers, Conferences [Proceedings]. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, USA, 11574–11583. DOI: https://doi.org/10.1109/CVPR.2019.01185
Reichenbach, M., Liebischer, L., Vaas, S., & Fey, D. (2018). Comparison of Lane Detection Algorithms for ADAS Using Embedded Hardware Architectures. In Institute of Electrical and Electronics Engineers, Conferences [Proceedings]. Conference on Design and Architectures for Signal and Image Processing (DASIP). Porto, Portugal, 48–53. DOI: https://doi.org/10.1109/DASIP.2018.8596994
Sanders, J., & Kandrot, E. (2010). CUDA by Example: An Introduction to General-Purpose GPU Programming. Addison-Wesley Professional.
Selim, E., Alci, M., & Uğur, A. (2022). Design and implementation of a real-time LDWS with parameter space filtering for embedded platforms. Journal of Real-Time Image Processing, 19(3), 663–673. DOI: https://doi.org/10.1007/s11554-022-01213-3
Silva, G., Batista, D., Tosin, M., & Melo, L. (2020). Estratégia de detecção de faixas de trânsito baseada em câmera monocular para sistemas embarcados. Sociedade Brasileira de Automática, Anais do 23º Congresso Brasileiro de Automática [Anais]. Congresso Brasileiro de Automática, Campinas, Brasil.
Silva, G. B. (2021a). Cuda-based real-time ego-lane detection in embedded system-tusimple #0601. https: //youtu.be/1q%5C_Jy8MJjFw
Silva, G. B. (2021b). Método heterogêneo de detecção de faixas de trânsito em tempo real para sistemas embarcados. [Dissertation Master’s thesis, State University of Londrina]. Digital Library. http:// www.bibliotecadigital.uel.br/document/?code=vtls 000236022.
Singh, S. (2015). Critical reasons for crashes investigated in the national motor vehicle crash causation survey (Traffic Safety Facts Crash Stats Report DOT HS 812 115). National Highway Traffic Safety Administration.
Sivaraman, S., & Trivedi, M. M. (2013). Looking at vehicles on the road: A survey of vision-based vehicle detection, tracking, and behavior analysis. IEEE Transactions on Intelligent Transportation Systems, 14(4), 1773–1795. DOI: https://doi.org/10.1109/TITS.2013.2266661
Son, J., Yoo, H., Kim, S., & Sohn, K. (2015). Real-time illumination invariant lane detection for lane departure warning system. Expert Systems with Applications, 42(4), 1816–1824. DOI: https://doi.org/10.1016/j.eswa.2014.10.024
Son, Y., Lee, E., & Kum, D. (2019). Robust multi-lane detection and tracking using adaptive threshold and lane classification. Machine Vision and Applications, 30, 111–124. DOI: https://doi.org/10.1007/s00138-018-0977-0
Wang, Y., Teoh, E., & Shen, D. (2004). Lane detection and tracking using b-snake. Image and Vision Computing, 22, 269–280. DOI: https://doi.org/10.1016/j.imavis.2003.10.003
Wu, C.-B., Wang, L.-H., & Wang, K.-C. (2019). Ultra-low complexity block-based lane detection and departure warning system. IEEE Trans on Circuits and Systems for Video Technology, 29(2), 582–593. DOI: https://doi.org/10.1109/TCSVT.2018.2805704
Yenikaya, S., Yenikaya, G., & Düven, E. (2013). Keeping the vehicle on the road: A survey on on-road lane detection systems. ACM Computing Surveys, 46(1). DOI: https://doi.org/10.1145/2522968.2522970
Yonglong, Z., Kuizhi, M., Xiang, J., & Peixiang, D. (2013). Parallelization and Optimization of SIFT on GPU Using CUDA. In Institute of Electrical and Electronics Engineers, Conferences [Proceedings]. 10th International Conference on High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing. Zhangjiajie, China. DOI: https://doi.org/10.1109/HPCC.and.EUC.2013.192
Yu, Y., & Jo, K.-H. (2018). Lane detection based on color probability model and fuzzy clustering. In H. Yu & J. Dong (Eds.), Ninth International Conference on Graphic and Image Processing (ICGIP 2017). [Proceedings]. Washington, USA. DOI: https://doi.org/10.1117/12.2302941
Zeng, H., Peng, N., Yu, Z., Gu, Z., Liu, H., & Zhang, K. (2015). Visual tracking using multi-channel correlation filters. In Institute of Electrical and Electronics Engineers, Conferences [Proceedings]. IEEE International Conference on Digital Signal Processing (DSP), Singapore. DOI: https://doi.org/10.1109/ICDSP.2015.7251861
Zhang, Z., & Ma, X. (2019). Lane Recognition Algorithm Using the Hough Transform Based on Complicated Conditions. Journal of Computer and Communications, 7(11), 65–75. DOI: https://doi.org/10.4236/jcc.2019.711005
Zheng, F., Luo, S., Song, K., Yan, C.-W., & Wang, M.-C. (2018). Improved lane line detection algorithm based on hough transform. Pattern Recognition and Image Analysis, 28(2), 254–260. DOI: https://doi.org/10.1134/S1054661818020049
Zhi, X., Yan, J., Hang, Y., & Wang, S. (2019). Realization of CUDA-based real-time registration and target localization for high-resolution video images. Journal of Real-Time Image Processing, 16(4), 1025–1036. DOI: https://doi.org/10.1007/s11554-016-0594-y
Zou, Q., Jiang, H., Dai, Q., Yue, Y., Chen, L., & Wang, Q. (2020). Robust lane detection from continuous driving scenes using deep neural networks. IEEE Trans on Vehicular Technology, 69(1), 41–54. DOI: https://doi.org/10.1109/TVT.2019.2949603
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Guilherme Brandão da Silva, Daniel Strufaldi Batista, Décio Luiz Gazzoni Filho, Marcelo Carvalho Tosin, Leonimer Flávio Melo
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
The Copyright Declaration for articles published in this journal is the author’s right. Since manuscripts are published in an open access Journal, they are free to use, with their own attributions, in educational and non-commercial applications. The Journal has the right to make, in the original document, changes regarding linguistic norms, orthography, and grammar, with the purpose of ensuring the standard norms of the language and the credibility of the Journal. It will, however, respect the writing style of the authors. When necessary, conceptual changes, corrections, or suggestions will be forwarded to the authors. In such cases, the manuscript shall be subjected to a new evaluation after revision. Responsibility for the opinions expressed in the manuscripts lies entirely with the authors.
This journal is licensed with a license Creative Commons Attribution-NonCommercial 4.0 International.
