Safebench is a benchmarking platform based on Carla simulator for evaluating safety and security of autonomous vehicles. The info flowchart of different modules is shown below:
The design of Safebench seperates objects and the policies that control objects to achieve flexible training and evaluation. Three importort features make Safebench different from existing autonomous driving benchmarks:
Safebench supports parallel running of multiple scenarios in one map, which dramatically increases the efficiency of colletcing training data and evaluation.
Safebench intergrates lots of scenario-generation algorithms, including data-driven generation, rule-based design and adversarial example.
This benchmark supports three types of running:
Train Agent: train policy model of autonomous vehicle with pre-defined scenarios.
Train Scenario: train policy model of scenario participants against pre-trained autonomous vehicles.
Evaluation: fix both autonomous vehicle and scenario to do evaluation.