Somewhere else, part 157

Some writings worth reading, here and there

Continue reading

Somewhere else, part 156

Some writings worth reading, found here and there

After more than a year of unsuccessful searching, authorities called in an elite group of statisticians. Working on their recommendations, the next search found the wreckage just a week later.

Continue reading

Selection_172

Computational Actuarial Science, with R

The book Computational Actuarial Science, with R is officially out. In the introduction of the book, and on the website of CRC, it is mentioned that the datasets can be found “in an R package on CRAN“, which is unfortunately incorrect. Some datasets are too large, so the package can not be uploaded on CRAN. Hopefully, Christophe host the package on his website.

> install.packages("CASdatasets", repos = "http://dutangc.free.fr/pub/RRepos/")

or

> install.packages("CASdatasets", repos = "http://dutangc.free.fr/pub/RRepos/", 
type = "source")

Here are the files :

Insurance datasets

A collection of datasets, originally for the book ‘Computational Actuarial Science with R’ edited by Arthur Charpentier (CAS with R). Now, the package contains a large variety of actuarial datasets.

Version: 0.9-8
Published: 2014-05-21
Author: Christophe Dutang
Maintainer: Christophe Dutang <christophe.dutang at ensimag.fr>
License: GPL-2 | GPL-3 [expanded from: GPL (≥ 2)]
NeedsCompilation: no

Downloads:

Reference manual: CASdatasets.pdf
Package source: CASdatasets_0.9-8.tar.gz
Package installation: go to this page
Windows binaries: r-release: CASdatasets_0.9-8.zip
OS X Snow Leopard binaries: r-release: CASdatasets_0.9-8.tgz
OS X Mavericks binaries: r-release: CASdatasets_0.9-8.tgz
Old sources: CASdatasets archive
Selection_048

Time for a Break

I am off. Officially off. I will be in the South of France for a week, and then in North Brittany for another week. I mean, I will spend time to work since one more time, I will take a break… with a co-author… but family style. Having fun with the kids during the day, discussing work (joint papers as well as courses) late in the evening.

But I will be offline !

Selection_169

Somewhere else, part 155

Some writings worth reading, discovered here, and there

Continue reading

Selection_148

iTunes’ Terms & Conditions

In a recent post (in French) I did plot the evolution of the number of pages of some legal document, published (and updated) over a century

 I had the feeling that the same pattern should be observed on Terms and Conditions documents, that seem to be longer and longer. In Small print that’s longer than George Orwell’s Animal Farm! HSBC gets wooden spoon for endless terms and conditions, it was mentioned that HSBC’s terms and conditions for a current account “came in at a heavyweight 34,162 words – almost 5,000 words longer than George Orwell’s classic Animal Farm“.

In order to get the evolution of the length of such documents, I’ve been using https://web.archive.org/web/ on iTunes’s Terms and Contents’ page(s). Copies can be found from January 2009 till now. But changes seem to be annual, only. Here is the evolution of word counts in those documents.

In order to compare with something else, I’ve been using stats from Shakespeare’s work. As mentioned in http://pensourceshakespeare.org, there are 884,421 total words in Shakespeare’s 43 works (i.e. on average 20567.93 per work).

Terms and Contents’ documents are as big as a play (or a book in the case of HSBC’s document). But the evolution is not exponential. If anyone has more studies on similar topics, I’d be glad to hear about it.

Selection_101

Somewhere else, part 154

Some writings worth reading, discovered here and there,

If someone asks, “How are you?,” I sigh, shrug, and say, “Busy, like everyone else.” If pressed, I will admit that I spent some time with my family—the way a Mormon might confess to having tried a beer, once. For more than 20 years, I have worn what Ian Bogost has called “the turtlenecked hairshirt.” I can’t help it; self-abnegation is the deepest reflex of my profession, and it’s getting stronger all the time…

Surely, the Catholic tradition of monastics and mendicants lies behind this tendency that I share with my profession, but there are other traditions at work here. As H. L. Mencken said, Puritanism is “the haunting fear that someone, somewhere, may be happy.” Happiness is worldliness, and idleness is sin: Work is an end in itself, as Max Weber observed in The Protestant Ethic and the Spirit of Capitalism. Likewise, there’s an old, unspoken commandment, “A professor shall not be seen mowing the lawn on weekdays.”

Continue reading

Selection_115

Social Media Mining and Bioinformatics (with R)

In June and July, I receive copies of two books,

For the first one, two recent interesting books deal with the same topic. Reza Zafarani, Mohammad Ali Abbasi and Huan Liu published last year Social Media Mining: An Introduction. Actually, the book can be downloaded from dmml.asu.edu. And – of course – there is Matthew A. Russell‘s Mining the Social Web. The main interest of this new book seems to be that it should be just perfect for R users !

Continue reading

Selection_103

Merchants of doubt, et le fonctionnement de la science

Il y a quelques jours, La Presse  mettait en ligne un texte intéressant, La BBC déclare la guerre à la « fausse science »

Les changements climatiques n’existent pas, les campagnes de vaccination sont dangereuses et votre téléphone cellulaire est en train de vous tuer à petit feu.

Il fut un temps où il était possible d’entendre ce genre d’opinions à la BBC. Des opinions scientifiques marginales, mais soutenues avec vigueur par certains chercheurs, et que le diffuseur britannique rapportait par souci « d’impartialité ». Mais cette ère est révolue.

Dans un document publié en juillet, la BBC prend une position très ferme sur la fausse science, qu’elle veut désormais évacuer de ses reportages.

« L’impartialité dans la couverture scientifique ne consiste pas simplement à faire état d’un large spectre de points de vue », écrit la BBC, qui croit qu’une application « trop rigide » des principes d’impartialité peut conduire à un « faux équilibre » dans les reportages.

La réflexion de la BBC a été suscitée par sa propre couverture des changements climatiques, qui a longtemps donné la parole aux climatosceptiques. Or, ceux-ci risquent maintenant d’avoir plus de difficulté à s’y voir offrir un micro.

Cet article m’a fait penser à un livre passionnant que j’avais lu l’automne dernier, Merchants of Doubt. Dans mon dernier compte rendu de lecture (ça faisait longtemps) je parlais de Big Data, a revolution that will transform how we live, work and think signé Viktor Meyer-Schönberger et Kenneth Cukier, et de The signal and the noise: Why Most Predictions Fail – but Some Don’t écrit par Nate Silver. Dans ce livre, on voyait comment on était passé d’une recherche de lien causal à une simple recherche de corrélation, dans des gros volumes de données. En fait, je ferais un parallèle avec le livre Merchants of Doubt, car comme l’expliquent Naomi Oreskes et Erik Conway, c’est précisément ce que combattent ce qu’ils appellent les marchants de doutes : tant qu’une relation causale n’est pas établie, le scientifique se devrait de douter (quelques pages – le premier chapitre – sont d’ailleurs en ligne). C’est cette idée que réfute  La BBC déclare la guerre à la « fausse science », et c’est l’histoire de ce principe (et des dérives qui ont suivi) que raconte ce livre.

Continue reading

Selection_082

L’échelle logarithmique expliquée aux juristes (et aux travailleurs)

Comme le rappelle https://sciencenews.org/blog/, il y a 400 ans, John Napier (que les français ont décidé d’appeler Neper) a ‘inventé’ le logarithme. Il publia son travail dans Mirifici logarithmorum canonis descriptio, et pour lui rendre hommage, le logarithme que l’on utilise le plus souvent est le logarithme népérien (qui est l’inverse de la fonction exponentielle).

Comme souvent, quand je parle de maths sur mon blog, on me reproche de ne m’adresser qu’à une élite (le commun des mortels dira des nerds) en employant un jargon incompréhensible. Pourtant, les logarithme sont présents partout !

Continue reading

Selection_068

Somewhere else, part 153

Some writings worth reading, found here and there,

Continue reading

Selection_015

The Pay-for-Performance Myth

Last week, Eric Chemi and Ariana Giorgi published an interesting article on “The Pay-for-Performance Myth

With all the public chatter about exorbitant executive compensation and income inequality, it’s useful to look at the relationship between chief executive officer pay and corporate performance. Typically, when the subject of their big pay packages arises, CEOs—usually through their spokespeople—say they are paid for performance. Does data back that up?

An analysis of compensation data publicly released by Equilar shows little correlation between CEO pay and company performance. Equilar ranked the salaries of 200 highly paid CEOs. When compared to metrics such as revenue, profitability, and stock return, the scattering of data looks pretty random, as though performance doesn’t matter. The comparison makes it look as if there is zero relationship between pay and performance.

In the article, they produce a copula-type plot (since ranks – only – are considered). Ariana kindly sent me the dataset (that was used in The Pay at the Top) to play with it

> base=read.table("ceo.csv",sep=";",header=TRUE)

Here I normalize (dividing by the size of the dataset) to have uniform distribution on the unit interval (instead of working with ranks, i.e. integers). If we remove that scaling factor, the scatterplot is that same as the one mentioned in  the Pay-for-performance myth.

> n=nrow(base)
> U=rank(base[,1])/(n+1)
> V=rank(base[,2])/(n+1)
> plot(U,V,xlab="Rank CEO Pay",
+ ylab="Rank Stock Return")

This is the copula type representation.

If we visualize the density of the copula (using the algorithm described in the joint paper with Gery and Davy), we get either

> library("copula")
> library("ks")
> library("MASS")
> library("locfit")
> n.res=32
> ctilde1=probtranscopkde(UVs,p=1,
+ u.out=seq(1/(2*n.res+1),1-1/(2*n.res+1),
+length=n.res),plots=TRUE)

Continue reading

An Open Lab-Notebook Experiment


Research blogs