Tag Archives: Elie

Reinforcement Learning in Economics and Finance, a state-of-the-art

Our joint paper, with Romuald Elie and Carl Remlinger entitled Reinforcement Learning in Economics and Finance just appeared in Computational Economics,

Reinforcement learning algorithms describe how an agent can learn an optimal action policy in a sequential decision process, through repeated experience. In a given environment, the agent policy provides him some running and terminal rewards. As in online learning, the agent learns sequentially. As in multi-armed bandit problems, when an agent picks an action, he can not infer ex-post the rewards induced by other action choices. In reinforcement learning, his actions have consequences: they influence not only rewards, but also future states of the world. The goal of reinforcement learning is to find an optimal policy — a mapping from the states of the world to the set of actions, in order to maximize cumulative reward, which is a long term strategy. Exploring might be sub-optimal on a short-term horizon but could lead to optimal long-term ones. Many problems of optimal control, popular in economics for more than forty years, can be expressed in the reinforcement learning framework, and recent advances in computational science, provided in particular by deep learning algorithms, can be used by economists in order to solve complex behavioral problems. In this article, we propose a state-of-the-art of reinforcement learning techniques, and present applications in economics, game theory, operation research and finance.

COVID19 pandemic control: balancing detection policy and lockdown intervention under ICU sustainability

After almost two months, with Romuald Elie, Chi (Tran Viet Chi) and Mathieu Laurière, we finally have a draft of the paper related to our recent work, entitled COVID-19 pandemic control: balancing detection policy and lockdown intervention under ICU sustainability” (available on HAL ArXiv and MedrXiv)

Our model is an extention of the classical SIR model, but more realistic for the COVID-19 (pronounded “cider”)

We use scenarios to see the impact on various quantities

but also optimal control to see the best strategy, when it comes to lockdown, and testing. All comments are welcome…

 

Reinforcement Learning in Economics and Finance

With Romuald Elie and Carl Remlinger we recently uploaded on ArXiv a paper on Reinforcement Learning in Economics and Finance

Reinforcement learning algorithms describe how an agent can learn an optimal action policy in a sequential decision process, through repeated experience. In a given environment, the agent policy provides him some running and terminal rewards. As in online learning, the agent learns sequentially. As in multi-armed bandit problems, when an agent picks an action, he can not infer ex-post the rewards induced by other action choices. In reinforcement learning, his actions have consequences: they influence not only rewards, but also future states of the world. The goal of reinforcement learning is to find an optimal policy — a mapping from the states of the world to the set of actions, in order to maximize cumulative reward, which is a long term strategy. Exploring might be sub-optimal on a short-term horizon but could lead to optimal long-term ones. Many problems of optimal control, popular in economics for more than forty years, can be expressed in the reinforcement learning framework, and recent advances in computational science, provided in particular by deep learning algorithms, can be used by economists in order to solve complex behavioral problems. In this article, we propose a state-of-the-art of reinforcement learning techniques, and present applications in economics, game theory, operation research and finance.

Optimal Claiming Strategies in Bonus Malus Systems and Implied Markov Chains

With Arthur David and Romuald Elie, we just wrote a short paper on bonus malus, and optimal strategies to claim a loss (or not)

In this paper, we investigate the impact of the claim reporting strategy of drivers, within a bonus malus system. We exhibit the induced modification of the corresponding class level transition matrix and derive the optimal reporting strategy for rational drivers. The hunger for bonuses induces optimal thresholds under which, drivers do not claim their losses. A numerical algorithm is provided for computing such thresholds and realistic numerical applications are discussed.

The paper is now online on http://papers.ssrn.com/id=2790583 and https://hal.archives-ouvertes.fr/hal-01326798.

Note that we do not discuss here legal issues here (in some contracts, it is compulsory to claim all losses, even small ones), but economic incentives and mathematical issues. Some popular journals in France did mention that issue, of non claims small losses (see http://leparticulier.fr/) but in those standard computations (see below), it is based on some naive model that we improve in our paper,

Mutualisation et Segmentation en Assurance

L’article Segmentation et Mutualisation, les deux faces d’une même pièce, coécrit avec Michel Denuit et Romuald Elie paraîtra dans les jours à venir, dans un numéro autour du big data en assurance, avec des articles de Patrick Thourot, sur la tarification du pay as you drive, de Lucie Taleyson, sur la tarification dans les assurances collectives, et tout plein d’articles passionnants, signés François-Xavier Hay ou encore Arnaud Chaput.

L’article (disponible en pdf) présente l’impact de la segmentation dans un environnement concurrentiel. Il d’un exemple simple, pour ne pas dire simpliste. Il sera perfectionné avec le pricing game organisé mi novembre. A suivre donc…

Segmentation et Mutualisation, les deux faces d’une même pièce ?

Ce billet est co-écrit avec Michel Denuit et Romuald Elie. Il s’agit de la version préliminaire d’un article qui sera bientôt soumis pour publication.

L’assurance repose fondamentalement sur l’idée que la mutualisation des risques entre des assurés est possible. Cette mutualisation, qui peut être vue comme une relecture actuarielle de la loi des grands nombres, n’ayant de sens qu’au sein d’une population de risques « homogènes » (Charpentier [2011]). Cette condition (actuarielle) impose aux assureurs de segmenter, ce que confirment plusieurs travaux économiques. Avec l’explosion du nombre de données, et donc de possibles variables tarifaires, certains assureurs évoquent l’idée d’un tarif individuel, semblant remettre en cause l’idée même de mutualisation des risques. Entre cette force qui pousse à segmenter et la force de rappel qui tend (pour des raisons sociales mais aussi actuarielles – ou au moins de robustesse statistique[1]) à imposer une solidarité minimale entre les assurés, quel équilibre va en résulter, dans un contexte de concurrence fort entre les compagnies d’assurance ?

Continue reading Segmentation et Mutualisation, les deux faces d’une même pièce ?