While Agathe just arrived in Vancouver for the Thirty-Eighth Annual Conference on Neural Information Processing Systems (also known as NeurIPS 2024), we just got the news that our paper Sequential Conditional Transport on Probabilistic Graphs for Interpretable Counterfactual Fairness was accepted at the 39th Annual AAAI Conference on Artificial Intelligence
Tag Archives: NeurIPS
Post-Calibration Techniques: Balancing Calibration and Score Distribution Alignment (NeurIPS’24)
Agathe Fernandes Machado will soon be on her way to Vancouver. She will attend the Thirty-Eighth Annual Conference on Neural Information Processing Systems (also known as NeurIPS 2024), to present a short paper on Post-Calibration Techniques: Balancing Calibration and Score Distribution Alignment
A binary scoring classifier can appear well-calibrated according to standard calibration metrics, even when the distribution of scores does not align with the distribution of the true events. In this paper, we investigate the impact of post-processing calibration on the score distribution (sometimes named “recalibration”). Using simulated data, where the true probability is known, followed by real-world datasets with prior knowledge on event distributions, we compare the performance of an XGBoost model before and after applying calibration techniques. The results show that while applying methods such as Platt scaling, Beta calibration, or isotonic regression can improve the model’s calibration, they may also lead to an increase in the divergence between the score distribution and the underlying event probability distribution.
Post-Calibration Techniques: Balancing Calibration and Score Distribution Alignment
Our paper Post-Calibration Techniques: Balancing Calibration and Score Distribution Alignment, will be presented at the NeurIPS workshop on Bayesian Decision-making and Uncertainty, that will take place in Vancouver, mid December. Agathe will be there to present it…