Discovering Approximate Functional Dependencies using Smoothed Mutual Information - CNRS - Centre national de la recherche scientifique Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Discovering Approximate Functional Dependencies using Smoothed Mutual Information

Résumé

We consider the task of discovering the top-K reliable approximate functional dependencies X → Y from high dimensional data. While naively maximizing mutual information involving high dimensional entropies over empirical data is subject to false discoveries, correcting the empirical estimator against data sparsity can lead to efficient exact algorithms for robust dependency discovery. Previous approaches focused on correcting by subtracting expected values of different null hypothesis models. In this paper, we consider a different correction strategy and counter data sparsity using uniform priors and smoothing techniques, that leads to an efficient and robust estimating process. In addition, we derive an admissible and tight bounding function for the smoothed estimator that allows us to efficiently solve via branch-and-bound the hard search problem for the top-K dependencies. Our experiments show that our approach is much faster than previous proposals, and leads to the discovery of sparse and informative functional dependencies. CCS CONCEPTS • Information systems → Data mining.
Fichier principal
Vignette du fichier
smoothed-info-article-without-ACM.pdf (932.44 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02880553 , version 1 (25-06-2020)

Identifiants

  • HAL Id : hal-02880553 , version 1

Citer

Frédéric Pennerath, Panagiotis Mandros, Jilles Vreeken. Discovering Approximate Functional Dependencies using Smoothed Mutual Information. KDD 2020 - 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Aug 2020, San Diego / Virtual, United States. pp.1254 -- 1264. ⟨hal-02880553⟩
179 Consultations
130 Téléchargements

Partager

Gmail Facebook X LinkedIn More