Tag Archives: Agathe

Agathe at AAAI’25, Philadelphia

Agathe Fernandes Machado is currently in Philadelphia, to present our paper Sequential Conditional Transport on Probabilistic Graphs for Interpretable Counterfactual Fairness, written with Ewen Gallic

In this paper, we link two existing approaches to derive counterfactuals: adaptations based on a causal graph, and optimal transport. We extend “Knothe’s rearrangement” and “triangular transport” to probabilistic graphical models, and use this counterfactual approach, referred to as sequential transport, to discuss fairness at the individual level. After establishing the theoretical foundations of the proposed method, we demonstrate its application through numerical experiments on both synthetic and real datasets.

Optimal Transport on Categorical Data for Counterfactuals using Compositional Data and Dirichlet Transport

Our recent paper, Optimal Transport on Categorical Data for Counterfactuals using Compositional Data and Dirichlet Transport, with Agathe and Ewen is now online

Recently, optimal transport-based approaches have gained attention for deriving counterfactuals, e.g., to quantify algorithmic discrimination. However, in the general multivariate setting, these methods are often opaque and difficult to interpret. To address this, alternative methodologies have been proposed, using causal graphs combined with iterative quantile regressions (Plečko and Meinshausen (2020)) or sequential transport Fernandes Machado et al. (2025)) to examine fairness at the individual level, often referred to as “counterfactual fairness.” Despite these advancements, transporting categorical variables remains a significant challenge in practical applications with real datasets.
In this paper, we propose a novel approach to address this issue. Our method involves (1) converting categorical variables into compositional data and (2) transporting these compositions within the probabilistic simplex of \mathbb{R}^d. We demonstrate the applicability and effectiveness of this approach through an illustration on real-world data, and discuss limitations.

https://freakonometrics.hypotheses.org/files/2025/01/transp2.png

See https://github.com/fer-agathe/transport-simplex for the codes

Post-Calibration Techniques: Balancing Calibration and Score Distribution Alignment (NeurIPS’24)

Agathe Fernandes Machado will soon be on her way to Vancouver. She will attend the Thirty-Eighth Annual Conference on Neural Information Processing Systems (also known as NeurIPS 2024), to present a short paper on Post-Calibration Techniques: Balancing Calibration and Score Distribution Alignment

A binary scoring classifier can appear well-calibrated according to standard calibration metrics, even when the distribution of scores does not align with the distribution of the true events. In this paper, we investigate the impact of post-processing calibration on the score distribution (sometimes named “recalibration”). Using simulated data, where the true probability is known, followed by real-world datasets with prior knowledge on event distributions, we compare the performance of an XGBoost model before and after applying calibration techniques. The results show that while applying methods such as Platt scaling, Beta calibration, or isotonic regression can improve the model’s calibration, they may also lead to an increase in the divergence between the score distribution and the underlying event probability distribution.

SCOR Project Newsletter #2

The second newsletter, related to the SCOR research project is now available. It is a brief summary of the second six months block, from April till the end of September (the first one is available here).

As explained, over the past six months, we have had several interns, including Noé Bosc-Haddad, Florent Crouzet, Julien Siharath, Ana María Patrón Piñerez and Cassandra Mussard, the visit of Laurence Barry and Fei Huang, Philipp Ratz and Samuel Stocksieker defended their PhD, François Hu finished his postdoctoral fellowship, while Marouane Il Idrissi and Arsene Zotsa just arrived, Agathe Fernandes Machado and Olivier Côté (co-supervised with Ewen Gallic and Marie-Pier Côté) finished their PhD courses and are now 100% on their research… We wrote papers, gave talks… Thank you to all those who have supported us, and continue to support us. At least two more years to work on insurance and predictive models, fairness, calibration, discrimination, trust, explainability, interpretability, market equilibria, competition, generative models, and so much more… We’ve still got a lot of work to do, and plenty of enthusiasm!

Probabilistic Scores of Classifiers, Calibration is not Enough

Our paper “Probabilistic Scores of Classifiers, Calibration is not Enough”, with Agathe Fernandes Machado, Emmanuel Flachaire, Ewen Gallic and François Hu is now available on https://arxiv.org/abs/2408.03421

In binary classification tasks, accurate representation of probabilistic predictions is essential for various real-world applications such as predicting payment defaults or assessing medical risks. The model must then be well-calibrated to ensure alignment between predicted probabilities and actual outcomes. However, when score heterogeneity deviates from the underlying data probability distribution, traditional calibration metrics lose reliability, failing to align score distribution with actual probabilities. In this study, we highlight approaches that prioritize optimizing the alignment between predicted scores and true probability distributions over minimizing traditional performance or calibration metrics. When employing tree-based models such as Random Forest and XGBoost, our analysis emphasizes the flexibility these models offer in tuning hyperparameters to minimize the Kullback-Leibler (KL) divergence between predicted and true distributions. Through extensive empirical analysis across 10 UCI datasets and simulations, we demonstrate that optimizing tree-based models based on KL divergence yields superior alignment between predicted scores and actual probabilities without significant performance loss. In real-world scenarios, the reference probability is determined a priori as a Beta distribution estimated through maximum likelihood. Conversely, minimizing traditional calibration metrics may lead to suboptimal results, characterized by notable performance declines and inferior KL values. Our findings reveal limitations in traditional calibration metrics, which could undermine the reliability of predictive models for critical decision-making.

Sequential Conditional Transport on Probabilistic Graphs for Interpretable Counterfactual Fairness

Our paper “Sequential Conditional Transport on Probabilistic Graphs for Interpretable Counterfactual Fairness“, written with Agathe Fernandes Machado and Ewen Gallic, is now online

In this paper, we link two existing approaches to derive counterfactuals: adaptations based on a causal graph, as suggested in Plečko and Meinshausen (2020) and optimal transport, as in De Lara et al. (2024). We extend “Knothe’s rearrangement” Bonnotte (2013) and “triangular transport” Zech and Marzouk (2022) to probabilistic graphical models, and use this counterfactual approach, referred to as sequential transport, to discuss individual fairness. After establishing the theoretical foundations of the proposed method, we demonstrate its application through numerical experiments on both synthetic and real datasets.

Talk in Stockholm, Sweden, at the Insurance Data Science Conference

This week, I will attend the Insurance Data Science conference in Sweeden. It has been a while… I was a keynote speaker at the one in London, ten years ago (to give a talk I still have feedbacks about – Getting into Bayesian Wizardry… (with the eyes of a muggle actuary) – by that time, the conference was “R in Insurance”), and then, we organized the one in Paris, back in 2017. Then we had the online events, but it was… different.

This time, I will get back to our recent paper A Sequentially Fair Mechanism for Multiple Sensitive Attributes, with François Hu and Philipp Ratz, and the equipy package, wrote with Agathe Fernandes-Machado and Suzie Grondin. The slides are available online.

Geospatial Disparities: A Case Study on Real Estate Prices in Paris

Our paper, Geospatial Disparities: A Case Study on Real Estate Prices in Paris, and Agathe Fernandes Machado, François Hu, Philipp Ratz and Ewen Gallic, is now online on ArXiv,

Driven by an increasing prevalence of trackers, ever more IoT sensors, and the declining cost of computing power, geospatial information has come to play a pivotal role in contemporary predictive models. While enhancing prognostic performance, geospatial data also has the potential to perpetuate many historical socio-economic patterns, raising concerns about a resurgence of biases and exclusionary practices, with their disproportionate impacts on society. Addressing this, our paper emphasizes the crucial need to identify and rectify such biases and calibration errors in predictive models, particularly as algorithms become more intricate and less interpretable. The increasing granularity of geospatial information further introduces ethical concerns, as choosing different geographical scales may exacerbate disparities akin to redlining and exclusionary zoning. To address these issues, we propose a toolkit for identifying and mitigating biases arising from geospatial data. Extending classical fairness definitions, we incorporate an ordinal regression case with spatial attributes, deviating from the binary classification focus. This extension allows us to gauge disparities stemming from data aggregation levels and advocates for a less interfering correction approach. Illustrating our methodology using a Parisian real estate dataset, we showcase practical applications and scrutinize the implications of choosing geographical aggregation levels for fairness and calibration measures.