Learning from an imbalanced distribution presents a major challenge in predictive modeling, as it generally leads to a reduction in the performance of standard algorithms. Various approaches exist to address this issue, but many of them concern classification problems, with a limited focus on regression. In this paper, we introduce a novel method aimed at enhancing learning on tabular data in the Imbalanced Regression (IR) framework, which remains a significant problem. We propose to use variational autoencoders (VAE) which are known as a powerful tool for synthetic data generation, offering an interesting approach to modeling and capturing latent representations of complex distributions. However, VAEs can be inefficient when dealing with IR. Therefore, we develop a novel approach for generating data, combining VAE with a smoothed bootstrap, specifically designed to address the challenges of IR. We numerically investigate the scope of this method by comparing it against its competitors on simulations and datasets known for IR.
This morning (Montréal time), Samuel Stocksieker defended his PhD thesis entitled “contribution of machine learning in modeling rare values and imbalanced data“. Cécile Capponi, Marianne Clausel, Julie Josse, Frédéric Planchet and Anne Sabourin, Christian-Yann Robert and Stéphane Loisel were in the jury,
the work is structured around major axes: Imbalanced Features and Imbalanced Regression. The first axis addresses the issue of feature imbalance, that is, when it concerns the attributes and not the variable to be explained. The first solution involves adjusting the distribution of a continuous covariate relative to a given target distribution. It proposes to combine weighted resampling and synthetic data generators. This strategy notably allows to deal with selection bias: when the distribution of the covariate in the training sample is significantly different from that of the population. A second solution is proposed in the context of multi-supervised learning, particularly with autoencoders. It relies on a new metric aimed at balancing the influence of variables during learning and is applicable not only to supervised and unsupervised models, but also to generative models such as variational autoencoders. The second part deals with the issue of regression from imbalanced data. Various preprocessing solutions, including synthetic data generation, are proposed. Initially, we propose to explore the initial data space by introducing new generators and methodologies to address the specific case of regression. We then propose to immerse the data in a latent space in order to provide a more conducive framework for synthetic data generation.
Cette semaine, Sam – Samuel Stocksieker – sera à Bordeaux, aux journées de statistiques, pour parler “smoothed bootstrap” et génération de données synthétiques pour la modélisation des extrêmes (papier co-écrit avec Denys Pommeret).
En apprentissage supervisé, il est assez fréquent de se retrouver confronté à des données présentant des distributions déséquilibrées. Cette situation entraîne souvent une difficulté d’apprentissage pour les algorithmes standards. La recherche et les solutions en matière d’apprentissage à partir de distributions déséquilibrées se sont principalement concentrées sur les tâches de classification. Malgré son importance, très peu de solutions existent pour la régression déséquilibrée (Imbalanced Regression). Dans cet article, nous proposons une procédure d’augmentation de données, nommée DENIS, basée sur des estimations à noyau de densité. Cette approche fournit une expression des densités conditionnelles des générateurs. Nous appliquons DENIS en régression déséquilibrée et proposons de le combiner à une nouveau type de générateur de type wild-boostrap pour simuler la variable cible, conditionnellement aux nouvelles données synthétiques. Nous évaluons les performances de l’algorithme DENIS dans des situations de régression déséquilibrée. Nous évaluons empiriquement et comparons notre approche et démontrons une amélioration significative par rapport aux techniques existantes.
"sendo l'intento mio scrivere cosa utile a chi la intende…"