Tag Archives: freakonometrics

Twitter

Popularité, ou reconnaissance ?

Leave a comment

Assez souvent, quand on est enseignant/chercheur, on remplit des demandes de financements, ou de promotions, on nous demande de parler de l’impact de nos travaux et de notre reconnaissance. La première fois que j’ai fait face à cette question, c’était il y a 5 ans, à Montréal, pour une demande de financement auprès du NSERC, au niveau fédéral. Sur le moment, la question m’a déstabilisé. Non pas qu’elle n’ait pas de sens, mais qu’on me la pose à moi. Qui suis-je pour savoir si ce que je fais a le moindre impact ?

Classiquement, l’impact, on le mesure avec les citations par exemple. On peut alors utiliser l’indice synthétique de Hirsch. Ces citations, elles commencent à figurer sur nos CV, ou tout du moins sur les CV en ligne que sont google scholar, orcid, research gate, academia,  etc. Un autre endroit où on retrouve la reconnaissance, et c’est probablement lié aux citations, ce sont les invitations en séminaire ou en conférence, les demandes pour faire partie de jurys, etc. Je devrais préciser que vu de l’extérieur, ce qu’on voit, ce sont les invitations “acceptées”. Les invitations en conférence sont probablement une information importante, mais là par exemple – pour des raisons personnelles – je me retrouve à décliner des invitations. Mon activité va baisser, mais comment l’interpréter ? Dans la liste, on devrait peut être rajouter les invitations pour arbitrer des publications d’articles dans des journaux et revues. On est alors très clairement identifié comme un pair sur un thème précis.

Mais pour comprendre en quoi nous avons un quelconque impact, il est toujours agréable quand on nous informe sur la raison qui a motivé une invitation. Malgré tout, c’est assez rare. Je pourrais toutefois noter quelques cas, où on nous invite pour présenter un papier particulier dans une conférence, ou quand l’éditeur explique brièvement la raison de la sollicitation, lors d’une demande d’arbitrage. Un exemple récent que je pourrais mentionner (histoire d’illustrer un peu mon billet) est l’invitation que j’avais reçu, en début d’année, pour aller parler à la Bank of England,

Plus précisément, comme le dit l’invitation (je reprends)

The Chief Economists Workshop is an annual forum that we host to facilitate high-level discussion between the Chief Economists from around thirty-five of the major central banks in the world, and academic and professional experts. The format is a mixture of presentations, discussion and debate. As such it offers a great opportunity for you to interact with and influence leading policymakers. There is no media presence. I attach a copy of the programmes from the previous two years to give you an idea of the format.

Many of us in the Bank of England are avid followers of your Twitter feed, and your ability to offer multi-disciplinary insights to current policy problems. I would like to invite you to give a talk along these lines – for example on actuarial and statistical insights into Big Data. It is not necessary to have a formal paper. The idea is more to discuss the issues and generate debate. We would be looking for a presentation of about 45 minutes followed by 45 minutes of Q and A and can schedule your talk to fit in with your commitments.

J’ai eu le plaisir d’être invité avec Andrew Gelman (NYU) et Andrew Lo (MIT). J’ai l’impression qu’un universitaire devrait être humilié de recevoir une telle lettre d’invitation (même si mon compte twitter est l’occasion de partager des informations en lien avec les thèmes de recherche qui m’intéressent – la statistique, l’économie, l’assurance, les risques, le climat, etc). Mais si on y pense deux minutes, la raison invoquée a l’avantage d’être honnête, et finalement flatteuse (on m’invite pour mon côté polymathe, ce qui me fait infiniment plaisir). Je doute que beaucoup de monde lise mes papiers (vu le prix qu’il faut payer pour y accéder, ça se comprendrait presque), et j’ai l’impression que quand on me demande de venir parler dans une conférence, c’est pour partager des connaissances, donner mon point de vue sur tel ou tel point, sur lequel j’ai pu réfléchir au cours de mes travaux de recherche.

Plus récemment, l’autre jour, j’étais particulièrement fier d’être dans les 10 scientifiques (médecins et scientifiques au sens large) les plus populaires sur Twitter, selon Science & Avenir,

D’autant plus que je n’avais rien demandé. Il y a plusieurs mois, j’avais dit – peut être un peu maladroitement – que je ne participerais plus à ces concours, à ces classements (en l’occurrence on nous invitait à participer à un répertoire des meilleurs blogs de sciences). Mais c’est toujours valorisant d’apprendre qu’on est “populaire”. Comme tous les chercheurs, on marche à la carotte, et avoir une médaille est une forme de reconnaissance qui fait forcément plaisir.

Ce n’est certes pas une reconnaissance par mes pairs, mais c’est malgré tout une forme de reconnaissance. Qui permet d’être interrogé (et cité) dans une enquête qui revient sur le financement de la couverture des catastrophes naturelles en France, une chronique sur les cat bonds, ou sur les investissements en un super-calculateur sur le climat. Cette reconnaissance en dehors du cercle classique (et très restreint) d’experts de sa propre discipline a du bon. Pour élargir son champ de vision tout d’abord et avoir un point de vue très différent. Mais aussi parfois rencontrer des gens en dehors du monde académique. Qui ne s’embarrassent pas à payer des éditeurs scientifiques pour avoir accès aux articles, mais qui sont d’avides lecteurs des blogs, qui ont souvent des données, ou des problèmes intéressants à nous soumettre. En revenant sur le financement du régime cat’nat en France pour les besoins de l’entrevue, on a commencé avec des collègues à retravailler sur la prévention. Personnellement, ce sont toujours ces discussions qui me permettent de trouver des thèmes de recherche (en plus de la lecture d’articles académiques finissant par un “tiens, il devrait être possible de regarder…” ou un “personnellement, j’aurais plus fait…”). Et j’adore ça ! peut être parce que c’est toujours l’occasion d’apprendre de nouvelles choses.

Academe, Conferences

Who interacts on Twitter during a conference (#JDSLille)

2 Comments

Disclamer: This is a joint post with Avner Bar-Hen, a.k.a. @a_bh, Benjamin Guedj, a.k.a. @bguedj and Nathalie Villa, a.k.a. @Natty_V2

Organised annually since 1970 by the French Society of Statistics (SFdS), the Journées de Statistique (JdS) are the most important scientific event of the French statistical community. More than 400 researchers, teachers and practitioners meet at each edition. In 2015, JDS took place in Lille, in France.

SFdS regularly tweets (with the account @Statfr) and for the first year a live-tweet was organized durind JdS. The Hashtag was #JDSLille. The aim of this post is a (brief) statistical analysis of the live-tweet.

Continue reading Who interacts on Twitter during a conference (#JDSLille) →

Academe, Internet

The role of blogging in academia

10 Comments

In a few days, I will participate to a panel discussion in Montréal, chaired by Marin Dacos, entitled  “Minor forms of academic communication: revamping the relationship between science and society?“, at the World Social Science Forum. I do not have much expertise  (compared with colleagues involved in the panel) even if I frequently observe the community of academic bloggers, and I regularly interact with some of them. For this panel discussion, Marin asked me to share my experience, as an academic blogger. So, let’s try to describe the Freakonometrics adventure…

  1. The origins: why and how the blog started?
  2. The practice: how do I blog?
  3. The future: why is it still worth blogging, in academia?

I will try to organize my post according to these three items (note that you can get directly to each of them if you want to skip some parts). But to be honest, my post will probably get soon very messy…

  • How blogging started – from experience at Univerisité de Rennes to ‘Freakonometrics’

The first version of the blog started at Université de Rennes 1, following a request from the IT department. In 2007 (as far as I remember), someone came up with the idea that all researchers should have a web page, or at least a page explaining their area of expertise, with links to papers, and lecture notes. But a lot of researchers were reluctant, and that IT person thought that blogs might be an interesting alternative. I was (extremely) skeptical, but since I just arrived in Rennes by that time, I thought it could be fun. I did have web pages for my courses (see a relic of the past, http://193.51.89.161/st/ for a course on time series I gave 10 years ago, in Paris), and I did have a webpage with weekly updates, that could be called a blog. So I did have some kind of experience. But still.

From a technical point of view, blog is a contracted form of weblog, which is a website made up of ongoing entries, that we will call posts. And those posts are published in reverse chronological order. So it makes it difficult to follows for students, unless they go on the blog frequently. There might be tags and categories, that can be used to distinguish posts related to conference, publications, and teaching. This first blog was a great experience. Teaching was fun, students did like the idea of the blog, and comments became a place to discuss. The blog was some kind of (open) forum, there were a lot of comments. I became blog addicted by that time.

Then I started to be recognized in conferences, and I wanted to stop having an eponymous blog (coincidence, or not, it was also by the time I moved to Montréal). Actually, several academic blogs are eponymous, it is rather common (we’ll get back on that later on). And that makes sense since most bloggers tend to identify their blogs, as both personal and professional. Consider for instance Ann Althouse, professor at University of Wisconsin Law School (since 1984) who met her husband through blogging (see http://nytimes.com/…). I guess you can find all here life (personal and professional) online. Boundaries between professional, academic and personal life may be difficult to establish, mainly because all those aspects intertwined constantly in the lives of scholars. After a almost three years, the Freakonometrics adventure started officially.

This blog is clearly an academic blog. Because of the editor, because of the contents, and because it is now hosted by hypotheses.org , a “platform for academic blogs in the humanities and social sciences“. It is one academic blog among many others1 . John Quiggin explained in 2006, “with the arguable exception of law, economics is the academic discipline where blogging has been embraced most enthusiastically“. This might explain why there is such an active – and enthusiastic – community (see more recently another discussion, by John Quiggin). About the academic blogosphere, Jacob Halford said that “the situation within academic blogging seems to be that we are currently a bunch of islands that are vaguely connected but not really arranged into continents and groups. We are all spread out across the digital world with a fragmented network between us.” What can we find in this community? Some economists like to use their name, like Greg Mankiw’s blog for instance (with subtitle “random observations for students of economics“). But most of them prefer to hide themselves behind a short title, like Confessions of a Supply-Side Liberal (by Miles Kimball), The Conscience of a Liberal (by Paul Krugman), or longer one, like Statistical Modeling, Causal Inference, and Social Science (by Andrew Gelman). The editor is never hidden, and we usually find a short bio, including a picture (most of the time), as well as a link to a webpage hosted by some university. Other examples might be I’m a bandit, with subtitle “random topics on optimization, probability and statistics. by Sébastien Bubeck“, or what’s new,  with subtitle “updates on my research and expository papers, discussion of open problems, and other maths-related topics. by Terence Tao“. Some blogs use puns (it is a feature that you can find on almost any blog: most of them use humor, just to explain that this is just a blog) like Hyndsight, “a blog by Rob Hyndman“. But the reference I like more is perhaps more one of those blogs where the (true) name appears, but only slightly…. In those blogs, the name of the editor appears only as the author of posts, like Marginal Revolution, where the contributors are  and , or Normal Deviate (with subtitle “Thoughts on Statistics and Machine Learning“) by Larry Wasserman (you simply have to click on the About hyperlink, but it is the only place you’ll find the name of the editor of the blog). Now, to explain the name of my blog, in a few lines, I should probably spend some time discussing a major influence.

  • How blogging started – Influences

In 2005, University of Chicago economist Steven Levitt and New York Times journalist Stephen Dubner published a collection of ‘economic’ articles, claiming that economics is, at root, the study of incentives. This is how http://freakonomics.com/blog/ started (the first post was published in September 2005). My blog is more about econometrics than economics, and I did borrow (not to say steal) the name freakonometrics to a colleague of mine at the Ecole Polytechnique. According to Francis Kramartz, there were two different approaches in econometric courses: see econometrics as an application of mathematical statistics, where the linear model is projection of the variable(s) of interest on the subset of linear combination of possible explanatory variables, and you derive properties, and then you discuss possible applications. Or you start with applications, with data, and then you try to find a (possibly) predictive model. Francis called this second approach freakonometrics. I find nice the maths behind econometrics (especially when you can mention geometry and projections), but I also love playing with datasets! I love the feeling you can have when you try to extract information, and think about visualization issues. I thought freakonometrics was a proper description of what will be in the blog.

Another important point when I started blogging was related to a so-called open community. I started blogging a few years after discovering R (a free software programming language, and a software environment, for statistical computing and graphics). The community of R users is based on the idea that we should share notes, codes, and tips. Since blogging is sharing some knowledge, it became natural to blog, including codes. And I have to confess that it has always been thrilling to see people willing to re-use what I’ve done in a blog post (as long as they don’t make money of it). Of course, there are alternative to blogging, such as being an active member on a forum (like stackoverflow), or answering to mailing list (see the paper by Timothy Stephen and Teresa Harrison on the Comserve experience). I truly admire contributors on forums or mailing lists. And somehow, we do the same kind of things. Except that on my blog, I am in charge.

  •  What the blog looks like ?

This might be a stupid question: since you read this post, you obviously can look around, and see how the blog looks like. On the left, I try to give a short description of my blog. I pretend that it is an “unpretentious academic blog“. It is an academic blog from its contents, not only because it is written by someone within academia. And it is unpretentious because I want my blog to be casual (but we’ll get back on this point later on). About me, I pretend to be “a surreptitious economist and born-again mathematician“. This is from my background. I did study Mathematics in France, then I discovered Economics. I got a master degree in Economics and Mathematics, and a PhD in Mathematics. Then I chose to join an Economics department, in France, for my first position, and finally got a position in a Mathematics department, in Montréal. Currently, I rediscover Mathematics, but I still love Economics. Econometrics in neither Mathematics, nor Economics. It happens to be somewhere in between. A macroeconomist will analyse and compare transportation prices. A microeconomist will try to understand why people decide to take their bike, or a car to go to work. They will try to explain why return tickets are usually cheaper than one-way tickets. An econometrician will try to get datasets with ticket price, for different dates, different destinations, etc, and then, try to quantify the price difference. Not necessarily explain it. This is what I do in my blog. I explain how to model, and I skip usually the interpretation part.

I claim that I am “a blog activist, and an actuary, too“. About the second part, in Europe, no one knows what actuarial science is, so usually, I do not mention it. In North America, it is much more popular. And yes, I am an actuary. I did publish books on mathematics of insurance, and I am about to edit a book on computational aspects of actuarial science. I am not proud of being an actuary, but the truth is, I find insurance problems puzzling and challenging, for mathematicians and economists. For the first part, yes, I keep writing on my blog that academics should blog. They do have legitimacy to comment and explain, so they should us it. I can even go on a conference on a holiday to talk about blogging !

On the right part of my blog, I try to get legitimacy. They are three different information,

I mention my academic publications since I believe they give me legitimacy, when I talk about econometric models, or probability. This is how academics judge other academics within academia.

I believe that those websites give me some credibility, not as an researcher, but as a blogger. I do also mention top site mentions, such as “100 Savvy Sites on Statistics and Quantitative Analysis“. I should probably mention that it might look like my academic profile gives me credibility as a blogger. But somehow, I have the feeling that the causality effect has changed: I now have credibility in my research because of my blog. Some editors asked me to refer some articles submitted because they read some posts on my blog (as they told me explicitly in their email). Some colleagues invited me because they know me2 from my blog, not from my research papers.

  • How do I blog, and what do I blog about ? 

My research and teaching activities are related to economics, mathematics, actuarial science, etc. I do blog about those subjects, using a less formal medium than academic journals, even if I write to an audience that I am usually talking to (students or researchers). And I cannot pretend that I write for non-economists, or non-mathematicians. Even if I want to, jargon comes naturally (even if I pretend to be casual). Different sorts of posts can be published on the blog. If we try to distinguish, there are

  • post to mention events and news
    • about forthcoming conferences, thesis prices, etc. This was done on the blog when I started, but this information is now shared with microblogging (through Twitter, via @freakonometrics). Nevertheless, there might be some recent examples on the blog, such as a PhD defense or a PhD price, or to mention the panel of the WSSF.
    • I should probably mention that I use micro-blogging to share readings I found interesting. Interesting tweets are now posted in a dedicated category, twice a week
  • posts to mention amusing situations to explain (possibly) difficult subjects
    • about game theory: when should I (otimaly) shoot at my son when playing with waterguns ? We have empty guns, we rush to fill them: the sooner I stop, the more likely he will get wet before I do (which is good, from my point of view), the longer I wait, the more water I will get, and lower the probabiliy to miss him. See also a discussion about optimal strategies to get married
    • about Markov Chains: what is the transition probability of a Markov Chain ? can I derive it in a simple case ? like Snake and Ladder game ? Here, I explained how to model that game, and how to simple Markov chains to see where you might be after rolling a dice ten or twenty time.

see also some post on genetic algorithm,

    • about demography issues: what is the age of the oldest person you know (from a TV commercial) ? How old are the popes,  or the Members of Parlements (twice actually)? Based on some open dataset, I compare the distribution of the age of MPs, and the distribution of the age of people who might be able to vote.

    • about probabililty: if I go the play the roulette, and I wish to maximize the probability of doubling my initial wealth, should I play small or should I play big ? or probability to win when playing cards : the more players there are, the longer the game? Some of the posts were published before I went to Las Vegas (for holidays).
    • about number theory, and complexity of strategies: on a Sunday evening game with kids (I pick a number, and you should find it), or a story about McGyver in an Afghan jail. In this post, I use a nice theoretical result in group theory in a McGyver story, trying to explain that yes, group theory can be fun, too.
    • about geometry: what could be the distance between points, and what are the connections with pigeonholes, is it simple to get your own Essher type graph, on breaking pieces of wood (part 1,  part 2 and part 3) or other technique to share a pizza fairly? Most of those posts are based on discussions with my kids, and I try to go further, and to illustrate difficult geometric concepts, or results, using simple discussions I had with my (young) kids
    • about magic tricks: how to sort matrix, per row. This post is based on the mathematical explanation of some magic trick.
    • about classification, and playing pétangue, following some discussions with students of mine, a few years back, where I suggested a dual version of a popular French sport,
    • about econometrics: based on Playmates’s measurements I try to explain the importance of working with individual observations (versus time related ones). Based on those measurements, I try to see if there is some correlation between chest and hip measurement, for young women. It turned out that there were no correlation at all, over 60 years, because we cannot treat those observations as individual independent observations, since there was a strong temporal evolution. Actually, chest and hip measurement had opposite trends with time, which tends to hide the true correlation

    • about stochastic processes, random walks (and option pricing): on the arcsine law, and drunkward’s walk. In those posts, I try to answer drunkward’s important question, and relate them to standard questions in mathematical finance
  • posts to react to articles, discovered online, somewhere else
    • about breaking records: how comes every year is the most expensive one, in terms of natural catastrophes, or about financial records. In this post, I try to see if over almost 20 years  it is an outstanding even to have 11 consecutive days were the index went down (and to compute that probability)

    • about the 100% chance of a nuclear incident called statitical certainty, about an article published in a French newspaper. In that article, two engineers explain that there is a 100% chance to have a nuclear accident, in France, over 30 years. Just by making a simple mistake in probability computation,

    • about e-cigarettes, and confusion in some French newspapper about a scientific study, where it was claimed that there was a difference, but (statistically) not significant.
     
    • about some probability of having an 11 hour match in tennis game, about the probability of having twice the same numbers in a lottery, or in UEFA series. In 2010, there was the longest tennis game, ever. And using some extreme value theory results, I try to estimate the odds of having such a game

    • about the traveling salesman (more a book review): inspired by William Cook’s “In Pursuit of the Traveling Salesman“, some codes were proposed to solve a (difficult) mathematical problem (in the context of collecting candies at Halloween)
    • about surveys, and pools: what does it mean if 75% of the people interviewed in a survey claim that they do not ‘believe’ in surveys, and opinion pools; and some code to predict the winner of some elections (based on several pools)
    • about insurance and bargaining: why is it – sometimes – rational for insurance company to bargain, with their insured, based on some old research paper, published in the 70’s.
    • about financial issues: what does that mean the a financial stock is hold, on average, 8 sec. ? what would a bunker full of gold be like, how large can it actually be. It started a discussion about the (difficult) estimation of what should be a (simple) average time
  • posts to discuss a question asked by a student, or a colleague, that puzzled me (it is then more a discussion, without answers)
    • about the interpretation of a parameter in a model: can weights in weighted least squares be understood as a frequency ?
    • about subadditivity and risk measures: why statements in discussion papers regarding Insurance company solvency might be incorrect, and yield to counterintuitive situations.
  • a presentation (and if possible an explanation) about a paradox,
    • about the Monty Hall paradox (see also a discussion about a similar topic on computing probability with respect to some information or more funny)
    • about Simpson’s paradox, and pies choice,
    • about bias selection: why we should not listen to students and policemen (see ) or why are there always more buses on the opposite side of the road.
    • about probabilities, like nuns and Hell’s angels in an airplane, some nice puzzles, here and there, and probabilities to have brothers and sisters: do boys have more brothers, or more sisters?
    • about events that will occur at some infinite time, with a strictly positive probability
  • posts to share some experiences with students (or by myself) to investigate a model, a dataset, or a computer function
    • about airline tickets: when it is optimal to buy – online – an airline ticket. I did use a dataset mentioned in a study on a similar topic, and I try to explain that this question is related to some risk aversion measure: are we looking for the date where, on average, the price is the lowest, or a date were, with 90% chance, the price is the lowest?
    • about graphs: with Ewen Gallic, what are the connections among twitter accounts of Members of Parliament (in France). The idea was to learn how to play with Twitter API, and to get a nice visualization,

There was also a post on hours of tweets (where I tried to see how long I can survive away from Twitter).

    • about graphical functions: finding Waldo in a picture, using the red and white stripped shirt , or enclaves in maps. Discovering image treatment functions
    • about circular density estimation: how to make sure (for hourly data) that 23:50 is close to 00:10 ? that (for spatial data) that -170 degrees (west) is close to 170 degrees (east) ? with application on earthquake location, or calls to 911. Actually, for 911 calls, the first post was entitled “minuit, l’heure du crime“, where I did try to figure if there were more crimes at midnight (which is precisely the time of discontinuity),
http://freakonometrics.blog.free.fr/public/perso3/911-mois.gif
    • about textmining, and letter appearance in language (and books): including La Disparition, a book written in French with no E (how different is letter appearance probability, compared with the conditional one, when E is removed?). Discovering textmining functions.

see also text extraction from tweets,

    • about first names, in France, per year. Using counts of birth per first name, region, and year, I try to get visualizations of spatial and temporal patterns associated to some common first names,
    • about car speed, or car accident (based on some dataset I got). I try to study the links between the speed of two consecutive cars (following a discussion I had with my wife while I was driving, where I try to explain that if I drive too fast, it might be because the driver in from of me is driving too fast),
    • about sharing datasets: we did generate a dataset linking zip codes and spatial coordinates or with counts of births in France, per day.
  • posts with a more historical perspective on a theory I discuss in a course
    • about the history of extreme value theory:what is the story behind the Fisher-Tippett theorem, and the law of three types, with Gumbel, Weibull and Fréchet distribution ? Did those people (really) work on extreme values ? Who got which result ? See also the history of the return period concept.
    • about the Student t test: who is this Student, and what are the connections with Guiness ?
    • about the chi-square distribution: what did Peason discovered first, the chi or the chi-square distribution ?
    • about discounting: Leonardo Fibonacci and discounting
    • about the law of small number: why is the Poisson distribution so important ?
    • about financial market efficiency (in French): who said that assets prices could be modeled as random walks (and therefore are then ‘purely’ random) ? What is a martingale ? See also an application to temperature time series in Montréal.
    • about optimization: did Newton and Raphson (from the so called Newton-Raphson algorithm) really invented the gradient descent ?
  • codes, references and slides related to some courses, conferences or research papers

    • about demography: codes and tables generated for the Appendices of a book on Bodily Injury, Insurance and Legal issues,
    • about climate change: codes and graphs related to a talk given a Lyon on extremal events, and climate change.

  • a more general discussion, about science and dissemination (not to say teaching)
    • about teaching computer codes to kids: on hacker generation, and why kids should learn how to code
    • about research mythology: how journalist discuss research issues (based on two experiences)
    • about humanities versus sciences: visiting the Gugenheim museum in NYC versus MoMaths (Museum of Mathematics), with kids

 

  • How do I blog, and what do I not blog about ? 

Now, I should admit that I will not blog on all topics. For instance, I was involved in some discussions, where a student of mine asked interesting questions, about religion and education. I wanted to share things I’ve heard (actually, read), but a lawyer told me that I should avoid to do so. And I know there are things that I should not post on my blog if I do not want to ruin my career. So for legal issues, there are things that I will not write on my blog. And similarly, I try to avoid libel actions, so when I write that someone published something stupid in a post, or in an article, to try to say that as nicely as I can. Again, I claim that my blog is “unpretentious“, so I am not here to fight, nor to be preachy, just to have fun! As we’ll discuss in a couple of paragraphs, I do fight every morning on my bike, I do fight when I arrive at work. My blog is still a peacefull place, and I want to keep it like that…

Another reason why I will not blog about everything is that I do not have legitimacy to blog on everything. I mean, I know a bit of mathematics, but when I post something about a property in geometry, I feel like an impostor. Similarly when I write something about some history of some statistical concept, about regulation in insurance, about simple game theory result, etc. I try to publish on topics that are either related to my research, or to my teaching activities. Even tonight, when writing this post, it looks like a big fraud to me, and I find it extremely hard to write some exegesis about my blogging activity.

Finally, I should confess that I do not blog about everything because I try keep some ideas for my research, that will (hopefully) end up with a publication in a peer-reviewed journal. Blogging does not get much credit in an academic career, let’s be pragmatic… Blogging is not a substitute for other academic writings! But they can clearly coexist (see a discussion on insidehighered blog). If we compare a blog post and a (standard) academic article, it takes more time to write a paper, mainly because full referencing is necessary, because it is necessary to convince the editors, as well as referee(s) that you wrote something original, that is probably a major contribution. It is much faster to publish a post! And probably most important, while blogging, you can explore a question, you do not need to answer it.

  • How do I blog? Somewhere between an academic paper and a journalist article?

As mentioned previously, some academics do publish posts in blogs hosted by newspapers, such as The Conscience of a Liberal (by Paul Krugman). Somehow, those journals (here the New York Times) host academics the same way newspapers hosted some opinion pages, were academics where invited to give their point of view, a few years back. But, as John Quiggin explained in 2006, “newspapers are generally reluctant to repost on academic working papers and similar publications unless the conclusions are obviously newsworthly“. Economics in newspapers has to be related to macroeconomics, and science has to be related…. to medicine or technology (the science page is now a nice advertising page for the most recent smartphone applications, or the electronic cigarette, as mentioned previously). And, as mentioned by Robert Cottrell, a couple of decades ago, experts (not to say scholar) “have functioned as sources for newspaper journalists. Their opinions would emerge often mangled and simplified, always truncated, in articles over which they had no final control.” Now, with blogs, it is possible to read them directly, in a style that is easier to understand, compared with academic (standard) publications. “The general reader has access to expertise that was easily available, a decade ago, only to the insider or the specialist“. Writing a post is an academic blog is neither pretending being a journalist, nor writing an academic article. In the traditional process of research, we discuss with colleagues, possibly in conferences, but only the final publication remains. False starts and heuristics are skipped, because they might appear as un-necessary to get an understanding of the article. And I truly believe that this is exactly where blogging become interesting.

Why am I still blogging? and why I will probably do it for a long time…

  • Why am I still blogging ? a peaceful island within academia ?

One of the main reason why I am still blogging after 6 years, with enthusiasm, is because it is still a lot of fun. And it is a place that I appreciate all the more that academia is currently a nightmare. I might sound incredibly cynical (and I have to confess that I think I am becoming cynical), but I interact with the blogging community because I want to. I interact with students and colleagues because I have to. There are two important issues in academia, when talking about money: tuition fees rise, and the decrease of public funding for research. I have the feeling that (undergraduate) students, are consumers. And consumers are the real bosses, you know that, right? And as a professor, I am like the seller in the store. I can try to give some advice, but I am useless. Most of my students are no longer interested by the story behind a model, they want some recipes for their future job. They want to know what to use, and when. And since universities evaluate their professors, they ask students to fill evaluation forms. And professors do everything to get good evaluations. That is a simple and extremely rational game. And about the colleagues, I can tell you so many stories, that I have experienced, or heard about. Everyone is now suspicious. And again, it is rational. The less money there is to do some research, the more competition. It there are one or two grants in my field of research, in Québec, I have no more colleagues and friends, I have only competitors. This is not (only) a feeling I have, it is truly something that I observed. Many time, I have been with colleagues, and we’ve been working with doors closed, not because we did not want to get disturbed, but to avoid that someone still our idea. We’ve even been to work in some coffee, outside the university. With students, it is a commercial world, while with colleagues, it is a world of paranoia (most of the time for good reasons). I always find odd, when I fill a form for a grant, the section about my ‘main contributions‘. When I arrived in Canada, some colleagues were paternalistic (one more time for good reasons) and they told me that I had to mention the impact of my research. Like my research could save the world, or help to cure from some awful diseases… No, what I do is theoretical, probably useless, and I cannot expect too much. Unfortunately, I am not Alvin Roth (who proved that mathematics can save lives, literally). And yes, when I have three consecutive hours to do some research, it is probably because either I missed a course, or because I forgot about a meeting…

Compared with those (standard) academic activities, blogging is fun. Within the blogosphere, I do not see competition, just motivation and stimulation. You can interact with other bloggers, learn from them, and so far, it is still a pleasure to blog. Some bloggers claim that it is a shame that blogging is not recognized (formally) within academia, but I think it is actually a great opportunity. We do blog because we want to, it is not another required task. So it can still be fun… And when talking about the impact of my activities, I believe that my blog has much more impact than my teaching (in a class room) or my research.

  • Why am I still blogging ? who I am blogging for ?

I have to confess that I blog mainly for myself, in the sense that I do not want to have a readership waiting for me to post something everyday! Also in the sense that my blog gives me a complete freedom to talk about things I find fun, with a whole person style (and actually I do use my bog to develop my own “writing voice“, to use Jill Walker’s words), discussing about personal issues. I remember when I started blogging. At first, I thought that no one was reading my blog, and then friends, colleagues told me that they were. It is actually thrilling (not to say scary) to have 5,000 readers for a blog post, when you think about the number of readers of academic articles. As claimed by Brendan O’Connor, “blogs are a more effective medium for intellectual influence than journal articles“. Somehow, it looks like academic journals try to avoid exposure. I mean, publishing an article in the Journal of Narrowly Focused Hyper Specialized Field Studies is a great place to hide your research.

If I wanted to be provocative, I would say that research is a social activity, where we need to keep, and to create, interactions with various researchers, reading papers, keeping our mind open. On the other hand, blogging is definitively a personal activity. And since I am an old bear, usually reluctant about standard social activities, blogging is perfect for me.

As mentioned in some previous posts, I use my blog as a note book, to keep traces of ideas, codes. So yes, blogging is personal. But it is opened, and anyone can access it. So I use my blog to promote my work, and my scholarship. Using Melissa Gregg‘s quote, I see academic “blogging as conversational scholarship“. Blog are great to encourage conversation! Blogs are coffee house (in the sense of XVIIth Century, in England). In blogs, we connect to other blogs, using comments, reactions, and hyperlinks. But actually, Derek de Solla Price explained in 1963 (see e.g. Doug Horne’s paper) “the prototype of the modern scientific paper is a social device rather than a technique for accumulating quanta of information“. So having informal discussion is probably the best way to work, as an academic. This is also the idea of Diana Crane, “the growth of scientific knowledge is a kind of diffusion process in which ideas are transmitted from person to person“. Using blogs, we can develop and connect a network of various people, from PhD students to practitioners in the industry, as well as more experienced academics who might share common interests. The blog is read by students, former students, colleagues, probably the dean, and even the department secretary. My kids too, someday…

  • Why am I still blogging ? cost-benefit analysis

With a simple cost-benefit analysis, I will probably blog if benefits are more important than costs. One component is related to time issue: is blogging costing, or saving time? I receive frequently emails, asking for explanations on a technical question (from students, former students, or anyone actually) that will need a  detailed email answer. I still believe that a “reply to public” is possible, with a blog post. Similarly, while teaching, the same question is asked twice a year. The first year, I can write an answer in a blog post, and then, I can integrete it to my notes (blog posts can even be more interesting than lectures notes). I cannot believe that blogging is a waste of time, since I see my blog as a long-term memory (I do have an extremely short-term memory, unfortunately). And just to be naughty, I do see a lot of academics that “do not have time to waste blogging” who can write extremely long, detailed (and most of the time, nicely structured) replies. If I write a detailed reply to a specific question (because I found the question interesting) I find it stupid not to share it. All the more because other people might also be interested in interacting… With blogs, dissemination is immediate, as well as comments and feedback.

A lot of researchers within academia still reject blogs because they’re not serious, and not peer reviewed. Not serious, I can live with it. I am a big fan of the Ig Nobels prices: yes, we can do serious research without being too serious. But rejecting blogs because they are not peer reviewed… you’ve got to be kidding me! Comments are open, and unless you want to sell Louis Vitton bags or Viagra, I publish all of them. In blog, comments can be more constructive than comments you get from referees in a so-called peer reviewed journal. Blog posts are published on the (open) web, not in some journal so expensive that no one can actually read it. Yes, commenting is not a formal or rigorous as a peer review publication, but having opened comments may contribute to establish quality and credibility of a blog. Having comments from the community is a great benefit. Further, blogging is interesting since I believe it did improve my teaching. In the sense that writing posts helped me (many times) to clarify my ideas. And I believe that my lectures are then better. So I guess I will keep blogging for a long long time…

I guess I will stop here, even if I might have tons of other things I would like to add. Based on this post, I now have think of something interesting to share in the panel, next week. But I also have to keep in mind “academic blogging can be an important medium, when it avoids the meta-narcissistic onanism of blogging about how important academic blogging is”, as claimed by Chris Parr.

1. To mention only some of them, see Edwin Chen’s http://blog.echen.me/, Michael Giberson and Lynne Kiesling’s http://knowledgeproblem.com/, Christopher Long’s http://angrystatistician.blogspot.ca/, Josh Hendrickson’s http://everydayecon.wordpress.com/, Rob Hyndman’s http://robjhyndman.com/hyndsight/, Andrew Gelman’s http://andrewgelman.com/, Christian Robert’s http://xianblog.wordpress.com/, David Stern’s http://stochastictrend.blogspot.ca/, John Cook’s http://www.johndcook.com/blog, Greg Mankiw’s http://gregmankiw.blogspot.com/, Mark Thoma’s http://economistsview.typepad.com/, Tony Cookson’s http://blog.thisyoungeconomist.com/, John Mount and Nina Zumel’s http://www.win-vector.com/blog/, John Myles White’s http://www.johnmyleswhite.com/, Bill McBrid’s http://www.calculatedriskblog.com/, Alex Singleton’s http://www.alex-singleton.com/, Liyun Chen’s http://blog.cloudlychen.net/, Eeshan Malhotra’s http://www.hotdamndata.com/, Percy Beach’s http://scepticalacademic.blogspot.ca/, Miles Kimball’s http://blog.supplysideliberal.com/, Brad DeLong’s http://delong.typepad.com/sdj/, Honglang Wang’s http://honglangwang.wordpress.com/, Matt Asher’s http://www.statisticsblog.com/, Cosma Rohilla Shalizi’s http://vserver1.cscs.lsa.umich.edu/~crshalizi/weblog/, Denis Haine’s http://denishaine.wordpress.com/blog-2/, Dimiter Toshkov’s http://rulesofreason.wordpress.com/, Christopher Gandrud’s http://christophergandrud.blogspot.ca/, Jodi Beggs’s http://www.economistsdoitwithmodels.com/, Eric Nguyen’s http://blog.datapunks.com/, Matt Bogard’s http://econometricsense.blogspot.ca/, Andrew Ziem’s http://heuristically.wordpress.com/, Dave Giles’s http://davegiles.blogspot.ca/, Jeff Ely and Sandeep Baliga’s http://cheaptalk.org/, Tyler Cowen and Alex Tabarrok’s http://marginalrevolution.com/, Kevin Bryan’s http://afinetheorem.wordpress.com/, Eran Raviv’s http://eranraviv.com/category/blog/, Gianluca Baio’s http://gianlubaio.blogspot.ca/, Ulrich Matter’s http://giventhedata.blogspot.ca/, Gregor Gorjanc’s http://ggorjan.blogspot.ca/, Nate Silver’s http://fivethirtyeight.blogs.nytimes.com/, Jesse Anttila-Hughes and Solomon Hsiang’s http://fight-entropy.com/, Francis Smart’s http://econometricsbysimulation.com/, Corey Chivers’s http://bayesianbiologist.com/, Steve Walker’s http://stevencarlislewalker.wordpress.com/,  Sebastien Bubeck’s https://blogs.princeton.edu/imabandit/, James Hamilton and Menzie Chinn’s http://www.econbrowser.com/

2. I won’t have time to discuss this point today, but I have been discussing with a lot of persons who truly believe that they know me because they read frequently my blog. And, to be honest, that might be true, and it is a strange felling. I mean, I remember some diners where people told me that they’ve been on my blog, and they remember what I did post, and I am like “great, but I don’t know you at all… I have never read any of your research papers, I don’t know what you might be working about…” This asymmetry put me in some awkward situations.

Twitter

Do you still have time to sleep ?

2 Comments

Last week, @3wen (Ewen) helped me to write nice R functions to extract tweets in R and build datasets containing a lot of information. I’ve tried a couple of time on my own. Once on tweet contents, but it was not convincing and once on the activity on Twitter following an event (e.g. the death of someone famous). I have to admit that I am not a big fan of databases that can be generated using standard function to study tweets. For instance, we can only extract tweets, notre-tweets (which is also an important indicator of tweet-activity). @3wen suggested to use

require("RJSONIO")

The first step is to extract some information from a tweet, and store it in a dataset (details can be found on https://dev.twitter.com/)

obtenir_ligne <- function(unTweet){
date_courante=unTweet$created_at
id_courant=unTweet$id_str
text=unTweet$text
nb_followers=unTweet$user$followers_count
nb_amis=unTweet$user$friends_count
utc_offset=unTweet$user$utc_offset
listeMentions=unTweet$entities$user_mentions
return(c(list(c(id_courant,date_courante,text,
nb_followers,nb_amis,utc_offset)),
list(do.call("rbind",lapply(listeMentions,
function(x,id_courant) c(id_courant,
x$screen_name),unTweet$id_str)))))
}

Now that we  have the code to extract information from one tweet, let us find several tweets, from one user, say my account,

nom="Freakonometrics"

The (small) problem here, is that we have a limitation: we can only get 100 tweets per call of the function

n=100
tweets_courants=scan(paste(
"http://api.twitter.com/1/statuses/user_timeline.json?
include_entities=true&include_rts=true&screen_name=
",nom,"&count=",n,sep=""),what = "character",
encoding="latin1")
tweets_courants=paste(tweets_courants[
1:length(tweets_courants)],collapse=" ")
tweets_courants=fromJSON(tweets_courants,
method = "C")

Then, we use our function to build a database with 100 lines,

extracTweets <- lapply(tweets_courants,
obtenir_ligne)
mentions=do.call("rbind",lapply(extracTweets,
function(x) x[[2]]))
colnames(mentions)=list("id","screen_name")
res=t(sapply(extracTweets,function(x) x[[1]]))
colnames(res) <- list("id","date","text",
"nb_followers","nb_amis","utc_offset")

The idea then is simply to use a loop, based on the latest id observed

dernier_id=tweets_courants[[length(
tweets_courants)]]$id_str

So, here we go,

compteurLimite=100

while(compteurLimite<4100){
tweets_courants=scan(paste(
"http://api.twitter.com/1/statuses/user_timeline.json?
include_entities=true&include_rts=true&screen_name=
",nom,"&count=",n,"&max_id=",dernier_id,sep=""),
what = "character", encoding="latin1")
tweets_courants=paste(tweets_courants[
1:length(tweets_courants)],collapse=" ")
tweets_courants=fromJSON(tweets_courants,
method = "C")

extracTweets <- lapply(tweets_courants[
2:length(tweets_courants)],obtenir_ligne)
mentions=rbind(mentions,do.call("rbind",
lapply(extracTweets,function(x) x[[2]])))
res=rbind(res,t(sapply(extracTweets,function(x) x[[1]])))
t(sapply(extracTweets,function(x) x[[1]]))
dernier_id=tweets_courants[[length(
tweets_courants)]]$id_str
compteurLimite=compteurLimite+100
}

resFreakonometrics=res=
data.frame(res,stringsAsFactors=FALSE)

All the information about my own tweets (and re-tweets) are stored in a nice dataset. Actually, we have even more, since we have extracted also names of people mentioned in tweets,

mentionsFreakonometrics=
data.frame(mentions)

We can look at people I mention in my tweets

gazouillis=sapply(split(mentionsFreakonometrics,
mentions$screen_name),nrow)
gazouillis=gazouillis[order(gazouillis,
decreasing=TRUE)]

plot(gazouillis)
plot(gazouillis,log="xy")
> gazouillis[1:20]
tomroud freakonometrics       adelaigue       dmonniaux
155              84              77              56
J_P_Boucher         embruns      SkyZeLimit        coulmont
42              39              35              31
Fabrice_BM            3wen          obouba          msotod
31              30              29              27
StatFr     nholzschuch        renaudjf        squintar
26              25              23              23
Vicnent        pareto35        romainqc        valatini
23              22              22              22

If we plot those frequencies, we can clearly observe a standard Pareto distribution,

Now, let us spend some time with dates and time of tweets (it was the initial goal of this post)… One more time, there is a (small) technical problem that we have to deal with: language. We need a function to convert date in English (on Twitter) to dates in French (since I have a French version of R),

changer_date_anglais <- function(date_courante){
mois <- c("Jan","Fév", "Mar", "Avr", "Mai",
"Jui", "Jul", "Aoû", "Sep", "Oct", "Nov", "Déc")
months <- c("Jan", "Feb", "Mar", "Apr", "May",
"Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec")
jours <- c("Lun","Mar","Mer","Jeu",
"Ven","Sam","Dim")
days <- c("Mon","Tue","Wed","Thu",
"Fri","Sat","Sun")
leJour <- substr(date_courante,1,3)
leMois <- substr(date_courante,5,7)
return(paste(jours[match(leJour,days)]," ",
mois[match(leMois,months)],substr(
date_courante,8,nchar(date_courante)),sep=""))
}

So now, it is possible to plot the times where I am online, tweeting,

DATE=Vectorize(changer_date_anglais)(res$date)
DATE=sapply(resSkyZeLimit$date,
changer_date_anglais,simplify=TRUE)

DATE2=strptime(as.character(DATE),
"%a %b %d %H:%M:%S %z %Y")
lt= as.POSIXlt(DATE2, origin="1970-01-01")
heure=lt$hour+lt$min/60
plot(DATE2,heure)

On this graph, we can see that I am clearly not online almost 6 hours a day (or at least not on Twitter). It is possible to visualize more precisely the period of the day where I might be on Twitter,

hist(heure,breaks=0:24,col="light green",proba=TRUE)
X=c(heure-24,heure,heure+24)
d=density(X,n = 512, from=0, to=24,bw=1)
lines(d$x,d$y*3,lwd=3,col="red")

or, if we want to illustrate with some kind of heat plot,

Note that we did it for my Twitter account, but we can also run the code on (almost) anyone on Twitter. Consider e.g. @adelaigue. Since Alexandre is tweeting in France, we have to play with time-zones,

res=extractR("adelaigue")
DATE=Vectorize(changer_date_anglais)(res$date)
DATE2=strptime(as.character(DATE),
"%a %b %d %H:%M:%S %z %Y",tz = "GMT")+2*60*60

or I can also look at @skythelimit who’s usually twitting from Singapore (I am in Montréal). I can seen clearly when we might have overlaps,

res=extractR("skythelimit")

Nice isn’t it. But it is possible to do much better… for instance, for those who do not ask specifically not to be Geo-located, we can see where they do tweet during the day, and during the night… I am quite sure a dozen posts with those functions can be written…

Miscellaneous

Ma vie et les bases de données

6 Comments

Ayant remarqué que les bloggeurs avaient souvent, un jour, envie de parler d’eux dans un billet, je vais faire pareil (oui, je sais Igor prétend que je ne fais déjà que ca ici). Je vais donc parler un peu de ma vie en tant que bloggeur.

Ce blog a été créé après un transfert du blog que je tenais à Rennes 1, qui avait été lancé voilà bientôt un peu plus de 2 ans, et je dois avouer (maintenant il y prescription) que j’ai honteusement volé le nom à un extrait de conversation que j’avais avec Francis Kramarz à l’X (reprenant le nom inventé par Stephen Dubner et Steven Levitt, i.e.freakonomics). L’idée du blog était de parler de choses qui m’amusent dans ma vie d’enseignant-chercheur.

L’économie mathématique n’est pas très sexy, et n’intéresse que peu de monde, donc j’évite d’en mettre trop… par contre, j’aime aussi beaucoup les maths appliqués, les statistiques, et l’économétrie. Et l’avantage quand on travaille sur la modélisation, y compris en tant qu’enseignant, c’est qu’on peut illustrer un peu comme on veut. Personnellement je comprends mieux la notion d’optimum de Pareto quand on me parle de se servir de l’eau à la cantine (comme ici), plutôt que de me donner une définition abstraite.

Et il s’avère que ça a plutôt tendance à réveiller les élèves quand on dit que les processus VAR permettent de suivre les mensurations des demoiselles de playboy (comme ici), en tous les cas plus que la modélisation des taux d’intérêt dans la zone euro. Les processus aléatoires, comme  je m’efforce à le dire (ici oulà), c’est beaucoup plus marrant appliqué à des problèmes d’alcooliques qu’appliqué à la valorisation d’options financières. Et les chaînes de Markov appliquées à l’analyse de la pluie en Bretagne (comme ici), ça intéresse tous les élèves qui viennent à la fac en vélo, ou sinon je peux aussi parler de consommation de papier toilette dans les toilettes publique (comme là).

Bref, peu importe le sujet, autant que ce soit amusant, l’idée étant d’illustrer une méthode, une technique, un concept abstrait…

Donc pour amuser mes étudiants (et mes lecteurs) je traîne beaucoup sur internet à la recherche de bases de données, afin d’illustrer des méthodes d’estimation. Alors pour ceux qui n’en ont jamais manipulé, les données, c’est un truc assez incroyable…. Vu de loin, ça ne ressemble pas à grand chose, genre ça peut ressembler à ça,

Mais il y a quelques années, on a vu arriver le datamining, et “les gens” ont commencé à retenir une idée un peu générale du genre “vos données valent de l’or”, et depuis, certains refusent de communiquer leurs données. Je ne parle pas de chercheurs qui ont collecté des données, et qui n’ont pas fini de publier dessus (je comprends qu’il puisse exister une relative compétition entre chercheurs), mais je parle de base qui existent…. mais que personne n’exploite. En fait “les gens” ont oublié que s’ils avaient de l’or, il faut peut être des mineurs pour aller le chercher, puis des joaillers pour le retravailler, et enfin des bijoutiers pour rendre ça présentable. Et c’est souvent agaçant de faire face à un refus, de voir ces belles données qui existent, mais qu’on ne peut avoir….

Bref, aujourd’hui je vais faire un billet sur les billets que j’aurais rêvé de faire, mais que je ne fais pas, faute de données….

  • L’argument (magique) de la CNIL pour les bases informatiques

Parmi les arguments qui ressortent sans cesse quand on demande des données, c’est “il faut avoir l’accord de la CNIL”. Bon, c’est l’argument de mauvaise foi par excellence…. On m’a sorti cet argument dans un hôpital, ou je voulais avoir des données sur les accouchements, connaitre la durée de la grossesse en fonction de caractéristiques de la mère (et du nombre d’enfants déjà eu, par exemple). Un hôpital m’a aussi dit “on peut vous ouvrir nos armoires, et vous pourrez noter vous même car ca n’est pas informatisé” (oui oui, j’ai eu ca… et j’ai pu vérifier avec la naissance de ma troisième: tout était sur papier… oups, j’ai dit de quel hôpital il s’agissait).

  • On veut la conclusion de l’étude avant de fournir les études

Toujours sur mes études sur la durée des grossesses, j’ai appris que l’Inserm constituait une grosse base de données. J’ai demande à y avoir accès, et fort gentiment, on m’a demande de déposer un dossier de recherche. J’ai donc mis l’objet de l’étude, le nom des étudiants qui auraient travaille sur la base, une biblio pour expliquer que je ne partais pas à l’aveugle, et la liste des variables qui m’auraient intéressées. Et la, par téléphone, on m’a dit que c’était très bien, mais “on a besoin d’avoir vos conclusions avant de vous donner accès aux données”. La trouve la démarche (supposée scientifique) surprenante. L’argument m’a un peu dépassé, et je n’ai pas donne suite.

  • L’invention d’arguments (farfelus, parfois)

Toujours sur mon étude sur les naissances, j’ai voulu avoir l’information à la source, c’est à dire dans les CAF. Elles connaissent le terme prévu, le terme réel entre autres, elles disposent de tout plein de variables explicatives. Bref, lasource de données idéale. J’ai donc contacté une vingtaine de CAF (car les CAF sont départementales), et j’ai eu une vingtaine de réponses différentes allant du “on n’a pas ce genre de données”, “on n’as pas le temps”,  “c’est la Caisse Nationale qui a ces données”, “il faut l’accord de la Caisse Nationale “… Bref, assez souvent, on botte en touche… Un peu comme “mais vous savez, on ne vend pas de fleurs” que m’a sorti Interflora (je voulais savoir si on commandait plus de fleurs a la Saint Valentin ou a la fête des mères, en vain).

  • Nous on fait dans le qualitatif, pas le quantitatif

Il y a quelques années, j’avais vu passer une étude sur l’évolution des mensurations de miss America (et les données sont en ligne ici). Trouvant l’analyse intéressante, j’ai contacte le comité Miss France pour avoir les mensurations des miss (et si possible des prétendantes), et Madame de Fontenay (oui oui, elle m’a écrit) m’a répondu “nous n’avons pas ces informations, les miss ne sont pas choisies pour leurs mensurations mais pour leur personnalité” (je n’invente presque pas – la seule invention vient de la retranscription car j’ai perdu le mail avec tous mes transferts de boites, mais c’est ce qui était dit en substance). Finalement j’ai ressorti mes vieux playboy, ce qui a permis d’avoir encore plus d’observations d’ailleurs….

  • Ah oui, je vois très bien, mais demandez au service ETDDHRC qui s’en occupe

L’argument magique du “c’est pas moi c’est mon frère” pour reprendre La Fontaine (ou ma fille, je ne sais plus trop). Je l’ai eu sur des demandes anciennes, faites à plusieurs reprises (et liées à des sujets qui ont beaucoup fait parler dans la blogosphere)

  • il y a 4 ou 5 ans, j’ai eu accès à un très gros fichier d’expertises de véhicules, consécutives à des accidents de la route, plusieurs centaines de milliers d’accidents analysés… on avait voulu travailler avec des étudiants sur l’impact des radars automatiques, mais surtout sur la fraude à l’assurance… Sauf qu’avoir les accidents c’est bien, mais il fallait normer par le nombre de véhicules en circulation… j’ai donc demande à plusieurs reprises un accès au fichier des cartes grises pour savoir combien de véhicules étant en circulation (marque, modèle)… tous les services se sont renvoyés la balle… Et je n’ai jamais eu la base…
  • j’ai eu la même discussion il y a quelques mois sur les nouvelles plaques… le Ministère de l’Intérieur affiche (ici) les départements les plus demande (car on a maintenant le choix du numéro que l’on pose sur la plaque), mais on ne peut pas croiser le département de résidence et le département de la plaque…. Étude qui m’aurait amusé car j’ai un joli modèle à tester derrière…
  • sinon à la même période, je souhaite analyser l’impact de la taille des classes sur la réussite scolaire (j’ai fait 4 billets sur le sujets, ici ou là, mais sur des données israéliennes): j’ai demande au Ministère de l’Éducation Nationale à avoir accès aux résultats individuels (mais anonymes) d’échantillons du test national de CE1. En particulier, j’étais aussi intéresse par la réussite des enfants nés en janvier par rapport à ceux nés en décembre… Tous les services se renvoyaient la balle, et je n’ai jamais eu les données.

J’oublie de préciser, mais a chaque fois, je précise qu’il s’agit d’une étude académique, et que peux m’engager a ne pas diffuser les données.

  • Le service juridique veut que vous signez 10 pages de “disclosure”

Alors ça c’est le pire… c’est arrivé l’autre jour avec un copain actuaire. Je l’avais sollicité pour quelques données, des triangles de liquidation comme on dit. Bon, ce sont des données un peu sensibles, mais je ne demandais pas les montants de provisions constitues, juste des paiements. J’ai un “c’est bon de notre cote, par contre il faudra que tu signes le document joint”, à savoir un document juridique de 10 pages, qui devaient être signées par les présidents de l’Université de Rennes 1 et de l’Université de Montréal (car c’était pour donner un projet à mes étudiants). Ne voulant pas prendre de risques, j’ai laissé tombé….

  • Allez vous servir sur le site (en faisant 5346  requêtes)

Cet argument m’a été sorti par le Ministère de l’Écologie:. Je voulais connaître la liste de toutes les communes touchées par une catastrophe naturelle (et qui ont bénéficié d’un arrêté), “c’est sur le site internet, vous savez, on est moderne nous”. Cette information est publique (parue au Journal Officiel) mais je voulais une base. J’ai fait un code l’autre jour pour aller faire plusieurs milliers de requetes (sur les matchs de tennis, mais ca prend du temps… et ca n’est pas toujours simple car il faut aller decoder des fichiers html) heureusement, il y a toujours des copains qui ont ces donnees et qui sont prêt à envoyer un fichier xls dans l’heure).

  • On veut bien donner des bases, mais à des journalistes, pas à des universitaires

Pour la première fois, on évoquait la discrétion, mais dans l’autre sens. Classiquement c’est “on vous donne des chiffres, mais faut pas le dire”, mais là c’était “on veut bien les donner, mais faut que ça se sache…”. Heureusement, Twitter est là, et on finira par trouver un point d’entrée…. à suivre donc…
Fort heureusement, certains ont compris leur intérêt, et n’hésite pas à m’envoyer des bases si je leur demande. Ca fait une étude gratuite pour eux, et moi je me serais amusé un peu (ou fait travaillé mes étudiants). Mais ce n’est pas la majorité. En attendant, je suis devenu voleur de données. Beaucoup de données traînent sur internet, et en bricolant un peu on peut les récupérer… L’autre jour un copain (que j’avais sollicité pour aller chercher des bases en faisant automatiquement des requêtes sur un site) me demandait “elles sont à qui ces données ?”. Et je dois avouer que depuis je doute…. Moralité, je suis un peu surpris de l’étymologie du mot données, car rares sont ceux qui en donnent vraiment… et c’est dommage.