“Life insurers can predict when you’ll die with about 98% accuracy”

The other day, I was reading a rather interesting article, “How GLP-1s Are Breaking Life Insurance” (a title reminiscent of the dozens of articles over the past 35 years that have predicted the death of insurance), with this short paragraph, which echoed an article I had just published (in French), on segmentation in insurance. Life … Continue reading “Life insurers can predict when you’ll die with about 98% accuracy”

Discrimination by proxy (a real case study)

Yesterday, with Laurence Barry, we posted a blog post “Who benefits from data sharing?” explaining why data sharing, in insurance, could end mutualization. Actually, it can also be bad in the context of discrimination. Consider here the same dataset, with claim occurence, in a real insurance portfolio, library(InsurFair) library(randomForest) Consider a version of this dataset … Continue reading Discrimination by proxy (a real case study)

Fairness and discrimination, PhD Course, #4 Wasserstein Distances and Optimal Transport

For the fourth course, we will discuss Wasserstein distance and Optimal Transport. Last week, we mentioned distances, dissimilarity and divergences. But before talking about Wasserstein, we should mention Cramer distance. Cramer and Wasserstein distances The definition of Cramér distance, for , is while Wasserstein will be (also for ) If we consider cumulative distribution functions, … Continue reading Fairness and discrimination, PhD Course, #4 Wasserstein Distances and Optimal Transport

Interprétabilité et explicabilité (formalisé) des modèles prédictifs

Dans Confessiones, Saint Augustin écrivait quid est ergo tempus? si nemo ex me quaerat, scio; si quaerenti explicare velim, nescio que l’on traduit Qu’est-ce-que le temps ? Si personne ne me le demande, je le sais. Si je veux l’expliquer à qui me le demande, je ne le sais plus. Pour aller un peu plus … Continue reading Interprétabilité et explicabilité (formalisé) des modèles prédictifs

Interpretability and explainability of predictive models

In 400 AD, in his Confessiones, Augustine wrote quid est ergo tempus? si nemo ex me quaerat, scio; si quaerenti explicare velim, nescio that can be translated as What then is time? If no one asks me, I know what it is. If I wish to explain it to him who asks, I do not … Continue reading Interpretability and explainability of predictive models

From multinomial regression to binary classification on some Siamese data

There are two kinds of people in the world: people who think there are two kinds of people in the world and people who don’t (borrowed from Menand (2018)). Because things are always simpler when we face only binary choice, aren’t they? But consider here the case were multiple options are possible, and let us … Continue reading From multinomial regression to binary classification on some Siamese data

Some general thoughts on Partial Dependence Plots with correlated covariates

The partial dependence plot is a nice tool to analyse the impact of some explanatory variables when using nonlinear models, such as a random forest, or some gradient boosting.The idea (in dimension 2), given a model for . The partial dependence plot for variable is model is function defined as . This can be approximated, … Continue reading Some general thoughts on Partial Dependence Plots with correlated covariates

STT5100, quiz (régression de Poisson #1)

Pour la fin de la semaine, j’avais donné un petit quiz, basé sur la base suivante, qui donne le nombre de cyclistes à une intersection, à New York City, download.file("http://freakonometrics.free.fr/NYCVelo.RData","velo.RData") load("velo.RData") str(base) ‘data.frame’: 214 obs. of 7 variables: $ Date : chr "1-Apr-17" "2-Apr-17" "3-Apr-17" "4-Apr-17" … $ HIGH_T : num 46 62.1 63 51.1 … Continue reading STT5100, quiz (régression de Poisson #1)

On the “correlation” between a continuous and a categorical variable

Let us get back on the Titanic dataset, loc_fichier = "http://freakonometrics.free.fr/titanic.RData" download.file(loc_fichier, "titanic.RData") load("titanic.RData") base = base[!is.na(base$Age),]loc_fichier = "http://freakonometrics.free.fr/titanic.RData" download.file(loc_fichier, "titanic.RData") load("titanic.RData") base = base[!is.na(base$Age),] On consider two variables, the age (the continuous one) and the survivor indicator (the qualitative one) X = base$Age Y = base$SurvivedX = base$Age Y = base$Survived It looks like … Continue reading On the “correlation” between a continuous and a categorical variable

STT5100, quiz (régression logistique)

Je mets en ligne des éléments de correction pour le quiz donné lundi dernier, en lien avec la régression logistique. Quiz 1 – régression logistique (sur données individuelles) On disposait de la base suivante, avec 44 bières testées, et un jugement porté (bon, ou pas), download.file("http://freakonometrics.free.fr/Beer.RData","Beer.RData") load("Beer.RData") str(base) ‘data.frame’: 44 obs. of  7 variables:  $ … Continue reading STT5100, quiz (régression logistique)

Il était prévisible que Léonard DiCaprio ne survive pas au naufrage du Titanic

Vendredi dernier, j’ai mis en ligne un petit quiz pour prédire qui a survécu au naufrage du Titanic, en 1912. J’avais mis en ligne une sous-base avec quelques passagers loc_fichier = "http://freakonometrics.free.fr/titanic.RData" download.file(loc_fichier, "titanic.RData") load("titanic.RData") str(base)loc_fichier = "http://freakonometrics.free.fr/titanic.RData" download.file(loc_fichier, "titanic.RData") load("titanic.RData") str(base) et plus précisément Survived: Passenger survival indicator (1 if survived) Pclass: Passenger class … Continue reading Il était prévisible que Léonard DiCaprio ne survive pas au naufrage du Titanic

Function basis and regression

In the first part of the course on linear models, we’ve seen how to construct a linear model when the vector of covariates is given, so that is either simply (for standard linear models) or a functional of (in GLMs). But more generally, we can consider transformations of the covariates, so that a linear model … Continue reading Function basis and regression

Lasso Regression (home made)

Again, this post is related to my MAT7381 course, where we will see that it is actually possible to write our own code to compute Lasso regression, We have to define the soft-thresholding functionThe R function would be soft_thresholding = function(x,a){ sign(x) * pmax(abs(x)-a,0) }soft_thresholding = function(x,a){ sign(x) * pmax(abs(x)-a,0) } To solve our optimization … Continue reading Lasso Regression (home made)

On the conjugate function

In the MAT7381 course (graduate course on regression models), we will talk about optimization, and a classical tool is the so-called conjugate. Given a function its conjugate is function such that so, long story short, is the maximum gap between the linear function and . Just to visualize, consider a simple parabolic function (in dimension … Continue reading On the conjugate function