This algorithm runs at 48Hz on an NVIDIA 1070Ti and Intel i5 desktop, which is state-of-the-art result as of September 2018.
In self-driving, lane marking detection is an important task. In this project, I made use of (and modified) VPGNet as the neural network for lane detection with an input of a raw RGP picture. I wrote the computer vision algorithm for post-processing of the output from VPGNet and maps the results into a bird’s eye view. The final results are stored as a set of polylines, which can be fed into a vehicle controller.
Apart from feeding into controller, we also have a seprate workflow which utilizes the lane detection for HD Map creation. This requires high accuracy instead of high speed. Below is a demo of the performance of my algorithm when it comes to a demand for high accuracy.
The neural network makes use of VPGNet. Below is an output example directly from VPGNet, without post processing.
VPGNet runs in caffe. It is poorly documented. Also, redundancy is present in the network so we need some truncating. We trained the network on caltech dataset.
The post-processing for this project poses the greatest challenge. From the output (pixelwise) of neural network to a mathematical form of polylines, tons of work is required. I use houghlines algorithm, clustering algorithm, inverse perspective mapping (IPM), and a lot of filteing and adjustment to obtain the final results.
In order to achieve fast performance, I spent a lot of time optimizing the code.
I truncated the original version of VPGNet and customized it for our needs. The resulted network runs at ~68Hz.
I rewrote the most time-consuming part of the project in C++ and used swig for the interface between python and C++.
Also, I optimized the code in IPM (inverse perspective mapping) and reduced redundancy in code to the largest extent.
The final algorithm runs at 48Hz on an NVIDIA 1070Ti and Intel i5 desktop, which is state-of-the-art result as of September 2018.
The final code was delivered in the form of a standalone software package for the team to use. I wrote detailed documentation regarding its usage (python flags, etc.).