
Tomorrow, I will give a talk on “Balance and Calibration of Probabilistic Scores:“” From GLM to Machine Learning” at Singapore campus – ESSEC Asia-Pacific. The abstract is
This study evaluates binary classifier performance with a focus on calibration, which is often overlooked by traditional metrics like accuracy. In high-stakes domains such as finance and healthcare, well-calibrated probabilities are crucial. We highlight the limitations of standard calibration metrics, particularly under score distortions and heterogeneous distributions. To address this, we introduce the Local Calibration Score and advocate optimizing models using Kullback-Leibler (KL) divergence to better align predicted scores with true probabilities. Our approach emphasizes balancing global and local calibration, ensuring overall distributional alignment while maintaining reliability across different score ranges. Using Random Forest and XGBoost across diverse datasets, we show that KL-based tuning improves calibration without sacrificing performance. Our results reveal that relying solely on traditional metrics can mislead model assessment, especially in sensitive decision-making scenarios. This is some joint work with Agathe Fernandes Machado and Ewen Gallic.
OpenEdition suggests that you cite this post as follows:
Arthur Charpentier (February 1, 2026). Balance and Calibration of Probabilistic Scores: From GLM to Machine Learning. Freakonometrics. Retrieved February 13, 2026 from https://doi.org/10.58079/15lky