Probabilistic performance estimators for computational chemistry methods: The empirical cumulative distribution function of absolute errors - CNRS - Centre national de la recherche scientifique Accéder directement au contenu
Article Dans Une Revue Journal of Chemical Physics Année : 2018

Probabilistic performance estimators for computational chemistry methods: The empirical cumulative distribution function of absolute errors

Pascal Pernot
Andreas Savin
  • Fonction : Auteur
  • PersonId : 983184

Résumé

.Benchmarking studies in computational chemistry use reference datasets to assess the accuracy of a method through error statistics. The commonly used error statistics, such as the mean signed and mean unsigned errors, do not inform end-users on the expected amplitude of prediction errors attached to these methods. We show that, the distributions of model errors being neither normal nor zero-centered, these error statistics cannot be used to infer prediction error probabilities. To overcome this limitation, we advocate for the use of more informative statistics, based on the empirical cumulative distribution function of unsigned errors, namely, (1) the probability for a new calculation to have an absolute error below a chosen threshold and (2) the maximal amplitude of errors one can expect with a chosen high confidence level. Those statistics are also shown to be well suited for benchmarking and ranking studies. Moreover, the standard error on all benchmarking statistics depends on the size of the reference dataset. Systematic publication of these standard errors would be very helpful to assess the statistical reliability of benchmarking conclusions.
Fichier principal
Vignette du fichier
DOC.pdf (4.52 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-02010898 , version 1 (25-08-2022)

Identifiants

Citer

Pascal Pernot, Andreas Savin. Probabilistic performance estimators for computational chemistry methods: The empirical cumulative distribution function of absolute errors. Journal of Chemical Physics, 2018, 148 (24), pp.241707. ⟨10.1063/1.5016248⟩. ⟨hal-02010898⟩
45 Consultations
70 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More