Enhancing the Performance of Multi-Vehicle Navigation in Unstructured Environments using Hard Sample Mining

Yining Ma^1,2,3, Ang Li¹, Qadeer Khan^1,2, Daniel Cremers^1,2

Technical University of Munich¹, Munich Center for Machine Learning², The Konrad Zuse School of Excellence in Learning and Intelligent Systems³

Code arXiv

This project page contains:

Abstract

Contemporary research in autonomous driving has demonstrated tremendous potential in emulating the traits of human driving. However, they primarily cater to areas with well built road infrastructure and appropriate traffic management systems. Therefore, in the absence of traffic signals or in unstructured environments, these self-driving algorithms are expected to fail. This paper proposes a strategy for autonomously navigating multiple vehicles in close proximity to their desired destinations without traffic rules in unstructured environments.
Graphical Neural Networks (GNNs) have demonstrated good utility for this task of multi-vehicle control. Among the different alternatives of training GNNs, supervised methods have proven to be most data-efficient, albeit requiring ground truth labels. However, these labels may not always be available, particularly in unstructured environments without traffic regulations. Therefore, a tedious optimization process may be required to determine them while ensuring that the vehicles reach their desired destination and do not collide with each other or any obstacles. Therefore, in order to expedite the training process, it is essential to reduce the optimization time and select only those samples for labeling that add most value to the training.
In this paper, we propose a warm start method that first uses a pre-trained model trained on a simpler subset of data. Inference is then done on more complicated scenarios, to determine the hard samples wherein the model faces the greatest predicament. This is measured by the difficulty vehicles encounter in reaching their desired destination without collision. Experimental results demonstrate that mining for hard samples in this manner reduces the requirement for supervised training data by 10 fold. Moreover, we also use the predictions of this simpler pre-trained model to initialize the optimization process, resulting in a further speedup of up to 1.8 times.

Video Comparison of Baseline Model and Improved Models

The visualizations show the qualitative performance of our model (trained with hard data) in comparison with the baseline and the model trained with random data for different scenarios involving variable number of vehicles and obstacles. As can be seen, the baseline and the one trained with random data have many collisions in contrast to our model.

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Probability Density Function of Collision Rate

The plots show the normalized probability density function of trajectories at different collision rates for eight vehicle/obstacle configurations. It can be observed that the distribution of the baseline method trained with additional random data (red curve) is always to the right of the distribution of our method (blue curve). This is more pronounced as the number of vehicles in the scene are increased. This shows that the method trained with random samples has more occurrences with higher collision rates.

8 vehicles, 0 obstacle

8 vehicles, 1 obstacle

10 vehicles, 0 obstacle

10 vehicles, 1 obstacle

12 vehicles, 0 obstacle

12 vehicles, 1 obstacle

15 vehicles, 0 obstacle

20 vehicles, 0 obstacle

Robustness Analysis for Noise

The analysis below shows the model's robustness to noise induced either to the steering angle output or the position state at the input.

Steering Angle Noise

Imperfections in the kinematic modelling has the potential to introduce disturbances in navigation of vehicles at inference time. Hence, the steering command predicted by the GNN model may not lead to the desired action being executed. Therefore, to model this behaviour, we introduce noise into the steering angle output Δφ and measure how the the success-to-goal rate of the baseline model and our model is affected. The steering angle noise satisfies the Gaussian Distribution: N(0, α*|φ|+β), where α changes from 0 to 0.3 in steps of 0.1, |φ| is the absolute predicted steering angle from model and β is a fixed bias term to be 2°:

Num. of Vehicle	Num. of Obstacle	Baseline Model					Our Model
Num. of Vehicle	Num. of Obstacle	No Noise	0	0.1	0.2	0.3	No Noise	0	0.1	0.2	0.3
8	0	0.8227	0.8284	0.8213	0.8136	0.8022	0.8959	0.8951	0.8884	0.8801	0.8794
8	1	0.8103	0.8121	0.8063	0.7949	0.7819	0.8857	0.8821	0.8798	0.8711	0.8633
10	0	0.7007	0.7035	0.6944	0.6793	0.6661	0.8342	0.8363	0.8264	0.8231	0.8067
10	1	0.6938	0.6906	0.6804	0.6636	0.6468	0.8126	0.8116	0.8059	0.8024	0.7935
12	0	0.5806	0.5743	0.5652	0.5561	0.5331	0.7581	0.7556	0.7511	0.7376	0.7321
12	1	0.5604	0.5621	0.5518	0.5353	0.5203	0.7379	0.7341	0.7284	0.7208	0.7123
15	0	0.3688	0.3681	0.3586	0.3451	0.3351	0.6234	0.6241	0.6158	0.6056	0.5961
20	0	0.1552	0.1544	0.1501	0.1477	0.1417	0.3782	0.3791	0.3761	0.3697	0.3588

Besides the table, the figure below also shows the relative change in success-to-goal ratio when the variance of noise intensity controlled by α is increased from 0 to 0.3. All values are normalized in reference to the results for α = 0. The curves show the mean performance across all the vehicle/obstacle configurations reported in the table above for the baseline and our model trained on hard sampled data. The figure also depicts the standard deviation for each model shown by the shaded regions.

The results in this figure show that our model is not only relatively more robust to noise in the steering angle as depicted by a slower drop in success-to-goal ratio but also is more stable in its prediction across the different configurations as demonstrated by a lower standard deviation.

Position Noise

It might be the case that the sensor measurements may not be accurate and the model may in reality be at a position different from what the sensor readings suggest. To measure the performance of the baseline and our model in the face of such inaccuracies, we introduce to position noise ΔX at the model input. The position noise satisfies the Gaussian Distribution: N(0, α*|v|+β), where α changes from 0 to 0.3 in steps of 0.05, |v| is the absolute velocity of the ego vehicles and β is a fixed bias term to be 0:

Num. of Vehicle	Num. of Obstacle	Baseline Model							Our Model
Num. of Vehicle	Num. of Obstacle	0	0.05	0.1	0.15	0.2	0.25	0.3	0	0.05	0.1	0.15	0.2	0.25	0.3
8	0	0.8227	0.8195	0.7837	0.7373	0.6721	0.5946	0.5151	0.8959	0.8941	0.8793	0.8586	0.8246	0.7721	0.7161
8	1	0.8103	0.8046	0.7697	0.7215	0.6453	0.5749	0.4994	0.8857	0.8797	0.8709	0.8455	0.8159	0.7727	0.7169
10	0	0.7007	0.6878	0.6545	0.5881	0.5181	0.4391	0.3474	0.8342	0.8308	0.8153	0.7821	0.7438	0.6915	0.6261
10	1	0.6938	0.6741	0.6332	0.5756	0.4963	0.4171	0.3295	0.8126	0.8087	0.7959	0.7724	0.7281	0.6728	0.6126
12	0	0.5806	0.5652	0.5163	0.4487	0.3712	0.2927	0.2233	0.7581	0.7536	0.7311	0.6948	0.6452	0.5909	0.5255
12	1	0.5604	0.5469	0.5001	0.4333	0.3563	0.2824	0.2135	0.7379	0.7271	0.7077	0.6761	0.6321	0.5794	0.5124
15	0	0.3688	0.3554	0.3099	0.2533	0.1981	0.1412	0.1022	0.6234	0.6143	0.5901	0.5553	0.4995	0.4373	0.3741
20	0	0.1552	0.1481	0.1245	0.1008	0.0786	0.0589	0.0465	0.3782	0.3767	0.3549	0.3185	0.2841	0.2447	0.2031

Same as what we did for the steering angle noise, the figure below also shows the relative change in success-to-goal ratio for increased α for position noise from 0.05 to 0.3. All values are normalized in reference to the results for α = 0. The curves show the mean performance as well as the standard deviation (shaded region) across all the vehicle/obstacle configurations reported in the table above for the baseline and our model.

We can draw similar conclusions that our model is relatively more robust and more stable in comparison to the baseline model.

GNN Model Inference Runtime

Here we show the results of the average runtime per inference step of our model for 8 different vehicle/obstacle configurations. The tests were done both on a GeForce RTX2070 GPU and an Intel Core i7-10750H CPU.

Num. of Vehicle	8	8	10	10	12	12	15	20
Num. of Obstacle	0	1	0	1	0	1	0	0
Runtime on GPU (s)	0.00775	0.00788	0.00773	0.00804	0.00801	0.00807	0.00805	0.00806
Runtime on CPU (s)	0.00823	0.00874	0.01027	0.01144	0.01314	0.01391	0.01604	0.02440

BibTeX


      @misc{ma2024enhancingperformancemultivehiclenavigation,
        title={Enhancing the Performance of Multi-Vehicle Navigation in Unstructured Environments using Hard Sample Mining}, 
        author={Yining Ma and Ang Li and Qadeer Khan and Daniel Cremers},
        year={2024},
        eprint={2409.05119},
        archivePrefix={arXiv},
        primaryClass={cs.MA},
        url={https://arxiv.org/abs/2409.05119},}

Enhancing the Performance of Multi-Vehicle Navigation in Unstructured Environments using Hard Sample Mining

This project page contains:

Abstract

Video Comparison of Baseline Model and Improved Models

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Our Model (Baseline with additional Hard Data)

Baseline Model

Baseline with additional Random Data

Probability Density Function of Collision Rate

8 vehicles, 0 obstacle

8 vehicles, 1 obstacle

10 vehicles, 0 obstacle

10 vehicles, 1 obstacle

12 vehicles, 0 obstacle

12 vehicles, 1 obstacle

15 vehicles, 0 obstacle

20 vehicles, 0 obstacle

Robustness Analysis for Noise

Steering Angle Noise

Num. of

Vehicle

Num. of

Obstacle

Baseline Model

Our Model

No Noise

0

0.1

0.2

0.3

No Noise

0

0.1

0.2

0.3

8

0

0.8227

0.8284

0.8213

0.8136

0.8022

0.8959

0.8951

0.8884

0.8801

0.8794

8