The model retraining cycle
Here is a quick example of badly annotated data: Of course this is a person..no doubt (Taken from BD100K) — automatically detected by our system:
Our method identifies underrepresented, badly annotated, and out-of-distribution data, allowing us to capture difficult-to-simulate variability in the real-world data distribution. By leveraging generative AI techniques (such as style transfer) and combining them with other approaches in a systematic, product-level approach, we can develop faster, more robust, and accurate perception models better suited for real-world driving scenarios, ultimately improving the safety and performance of self-driving cars and guiding the development of advanced simulation tools.
Our analysis of the BDD100K Dataset revealed high ratios of incorrectly labeled data, with 7% for the “train” label, 4% for the “person” label and 8% for the car label. This can negatively impact model evaluation and results. Here are real-world examples from the dataset and our automatic identification of these cases.
Here we see how through our graph representation of the data, we can detect the various cases to automatically build a dataset.
Underrepresented examples:
By focusing on rare modalities and hard-to-find scenarios, we can use generative AI techniques to augment the dataset and provide additional coverage of the space of possible scenarios allowing us to capture difficult-to-simulate variability in the real-world data distribution. This is different from generating synthetic data from scratch. We use Style Transfer to find common modalities of the label, and transform them to the rate modality (People in the day vs people at night). This approach will ultimately lead to more robust and accurate perception models for self-driving cars that can handle a wide range of real-world scenarios and challenges, while also guiding the development of more advanced simulation tools.
Badly labeled data
Badly defined labels
Here we can see the case of badly defined labels. We can see that Drivable area has a lot of very similar images, in two different connected components. This indicates that the initial labeling requirements were improperly set.