DeepAbstraction: 2-Level Prioritization for Unlabeled Test Inputs in Deep Neural Networks - VERIMAG Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

DeepAbstraction: 2-Level Prioritization for Unlabeled Test Inputs in Deep Neural Networks

Résumé

Deep learning systems recently achieved unprecedented success in various industries. However, DNNs still exhibit some erroneous behaviors, which lead to catastrophic results. As a result, more data should be collected to cover more corner cases. On the other hand, a massive amount of data consumes more human annotators (oracle), which increases the labeling budget and time. We propose an effective test prioritization technique, called DeepAbstraction to prioritize the more likely error-exposing instances among the entire unlabeled test dataset. The ultimate goal of our framework is to reduce the labeling cost and select the potential corner cases earlier before production. Different from existing work, DeepAbstraction leverages runtime monitors. In the literature, runtime monitors are primarily used to supervise the prediction of the neural network. Then, monitors trigger a verdict for each prediction: acceptance, rejection, or uncertainty. Monitors quantify the acquired knowledge into box abstraction during the training. Each box abstraction contains instances that share similar high-level features. In the test part, the verdict of monitor depends in which box abstraction a test instance resides. Moreover, we study intensively where corner cases can reside in the feature space, either near-boundary regions or nearcentroid regions. The existing test prioritization techniques can only prioritize many near-boundary instances and a few nearcentroid instances. Nevertheless, DeepAbstraction can effectively prioritize numerous instances from both regions. Therefore, our evaluation shows that DeepAbstraction outperforms the state-ofthe-art test prioritization techniques.
Fichier principal
Vignette du fichier
aitest22.pdf (1.59 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03911812 , version 1 (23-12-2022)

Identifiants

  • HAL Id : hal-03911812 , version 1

Citer

Hamzah Al Qadasi, Changshun Wu, Yliès Falcone, Saddek Bensalem. DeepAbstraction: 2-Level Prioritization for Unlabeled Test Inputs in Deep Neural Networks. AITest 2022 - IEEE 4th International Conference On Artificial Intelligence Testing, Aug 2022, San Francisco, United States. pp.1-8. ⟨hal-03911812⟩
93 Consultations
111 Téléchargements

Partager

Gmail Facebook X LinkedIn More