Tag Archives: Sam

SCOR Project Newsletter #2

The second newsletter, related to the SCOR research project is now available. It is a brief summary of the second six months block, from April till the end of September (the first one is available here).

As explained, over the past six months, we have had several interns, including Noé Bosc-Haddad, Florent Crouzet, Julien Siharath, Ana María Patrón Piñerez and Cassandra Mussard, the visit of Laurence Barry and Fei Huang, Philipp Ratz and Samuel Stocksieker defended their PhD, François Hu finished his postdoctoral fellowship, while Marouane Il Idrissi and Arsene Zotsa just arrived, Agathe Fernandes Machado and Olivier Côté (co-supervised with Ewen Gallic and Marie-Pier Côté) finished their PhD courses and are now 100% on their research… We wrote papers, gave talks… Thank you to all those who have supported us, and continue to support us. At least two more years to work on insurance and predictive models, fairness, calibration, discrimination, trust, explainability, interpretability, market equilibria, competition, generative models, and so much more… We’ve still got a lot of work to do, and plenty of enthusiasm!

Contribution of machine learning in modeling rare values and imbalanced data

This morning (Montréal time), Samuel Stocksieker defended his PhD thesis entitled “contribution of machine learning in modeling rare values and imbalanced data“. Cécile Capponi, Marianne Clausel, Julie Josse, Frédéric Planchet and  Anne Sabourin, Christian-Yann Robert and Stéphane Loisel were in the jury,

the work is structured around major axes: Imbalanced Features and Imbalanced Regression. The first axis addresses the issue of feature imbalance, that is, when it concerns the attributes and not the variable to be explained. The first solution involves adjusting the distribution of a continuous covariate relative to a given target distribution. It proposes to combine weighted resampling and synthetic data generators. This strategy notably allows to deal with selection bias: when the distribution of the covariate in the training sample is significantly different from that of the population. A second solution is proposed in the context of multi-supervised learning, particularly with autoencoders. It relies on a new metric aimed at balancing the influence of variables during learning and is applicable not only to supervised and unsupervised models, but also to generative models such as variational autoencoders. The second part deals with the issue of regression from imbalanced data. Various preprocessing solutions, including synthetic data generation, are proposed. Initially, we propose to explore the initial data space by introducing new generators and methodologies to address the specific case of regression. We then propose to immerse the data in a latent space in order to provide a more conducive framework for synthetic data generation.

Generalized Oversampling for Learning from Imbalanced datasets and Associated Theory

Our paper, Generalized Oversampling for Learning from Imbalanced datasets and Associated Theory: Application in Regression, written with Samuel Stocksieker and Denys Pommeret, has been accepted for publication in TMLR (Transactions on Machine Learning Research)

In supervised learning, it is quite frequent to be confronted with real imbalanced datasets. This situation leads to a learning difficulty for standard algorithms. Research and solutions in imbalanced learning have mainly focused on classification tasks. Despite its importance, very few solutions exist for imbalanced regression. In this paper, we propose a data augmentation procedure, the GOLIATH algorithm, based on kernel density estimates and especially dedicated to the problem of imbalanced data. This general approach encompasses two large families of synthetic oversampling: those based on perturbations, such as Gaussian Noise, and those based on interpolations, such as SMOTE. It also provides an explicit form of such machine learning algorithms. New synthetic data generators are deduced. We apply GOLIATH in imbalanced regression combining such generator procedures with a new wild-bootstrap resampling technique for the target values. We evaluate the performance of the GOLIATH algorithm in imbalanced regression where we compare our approach with state-of-the-art techniques.

 

Présentation à Bordeaux, Journées de Statistique

Cette semaine, Sam – Samuel Stocksieker – sera à Bordeaux, aux journées de statistiques, pour parler “smoothed bootstrap” et génération de données synthétiques pour la modélisation des extrêmes (papier co-écrit avec Denys Pommeret).

En apprentissage supervisé, il est assez fréquent de se retrouver confronté à des données présentant des distributions déséquilibrées. Cette situation entraîne souvent une difficulté d’apprentissage pour les algorithmes standards. La recherche et les solutions en matière d’apprentissage à partir de distributions déséquilibrées se sont principalement concentrées sur les tâches de classification. Malgré son importance, très peu de solutions existent pour la régression déséquilibrée (Imbalanced Regression). Dans cet article, nous proposons une procédure d’augmentation de données, nommée DENIS, basée sur des estimations à noyau de densité. Cette approche fournit une expression des densités conditionnelles des générateurs. Nous appliquons DENIS en régression déséquilibrée et proposons de le combiner à une nouveau type de générateur de type wild-boostrap pour simuler la variable cible, conditionnellement aux nouvelles données synthétiques. Nous évaluons les performances de l’algorithme DENIS dans des situations de régression déséquilibrée. Nous évaluons empiriquement et comparons notre approche et démontrons une amélioration significative par rapport aux techniques existantes.

Generalized Oversampling for Learning from Imbalanced datasets and Associated Theory

Our paper, Generalized Oversampling for Learning from Imbalanced datasets and Associated Theory, writen with Denys Pommeret and Sam Stocksieker, is now available ArXiv.

In supervised learning, it is quite frequent to be confronted with real imbalanced datasets. This situation leads to a learning difficulty for standard algorithms. Research and solutions in imbalanced learning have mainly focused on classification tasks. Despite its importance, very few solutions exist for imbalanced regression. In this paper, we propose a data augmentation procedure, the GOLIATH algorithm, based on kernel density estimates which can be used in classification and regression. This general approach encompasses two large families of synthetic oversampling: those based on perturbations, such as Gaussian Noise, and those based on interpolations, such as SMOTE. It also provides an explicit form of these machine learning algorithms and an expression of their conditional densities, in particular for SMOTE. New synthetic data generators are deduced. We apply GOLIATH in imbalanced regression combining such generator procedures with a wild-bootstrap resampling technique for the target values. We evaluate the performance of the GOLIATH algorithm in imbalanced regression situations. We empirically evaluate and compare our approach and demonstrate significant improvement over existing state-of-the-art techniques.

AISTATS 2023

This week, Sam will be in Valencia (Spain) to present our work on Data Augmentation for Imbaladed Regression

In this work, we consider the problem of imbalanced data in a regression framework when the imbalanced phenomenon concerns continuous or discrete covariates. Such a situation can lead to biases in the estimates. In this case, we propose a data augmentation algorithm that combines a weighted resampling (WR) and a data augmentation (DA) procedure. In a first step, the DA procedure permits exploring a wider support than the initial one. In a second step, the WR method drives the exogenous distribution to a target one. We discuss the choice of the DA procedure through a numerical study that illustrates the advantages of this approach. Finally, an actuarial application is studied.

Data Augmentation for Imbalanced Regression

Our paper Data Augmentation for Imbalanced Regression, writen with Denys Pommeret and Sam Stocksieker, is now available on ArXiv.

In this work, we consider the problem of imbalanced data in a regression framework when the imbalanced phenomenon concerns continuous or discrete covariates. Such a situation can lead to biases in the estimates. In this case, we propose a data augmentation algorithm that combines a weighted resampling (WR) and a data augmentation (DA) procedure. In a first step, the DA procedure permits exploring a wider support than the initial one. In a second step, the WR method drives the exogenous distribution to a target one. We discuss the choice of the DA procedure through a numerical study that illustrates the advantages of this approach. Finally, an actuarial application is studied.