A few years ago, I did publish a post to visualize and empirical claims frequency in a portfolio. I wanted to update the code.
Tag Archives: frequency
Extracting information from a keyboard…
Yesterday, Baptiste published a post on “ethno-photography” (here). As he mentioned it, in Paris 8, they experience a real absence of serious cleaning of office equipment. He then shows the keyboard of the only computer they can use in the sociology department (for forty researchers),
Apart from the fact that everyone in France should be ashamed to see how much is spent in universities (which is the first information we have from that picture), we should also be able to guess in which langage people work in this department.
I considered three books (two in French, one in English) and I would like to see the frequency of each letter,
- Mauss, manuel d’éthnographie (here), 1926
- Durkheim, Livre II: Les croyances élémentaires in Les formes élémentaires de la vie religieuse (here), 1912
- Ferri, Criminal Sociology (here), 1896
Those three books are in rich text format, I just changed it to get text files… Then, it is easy to count appearance of letters. E.g. for Mauss,
> library(corpora) > textfile=scan("MAUSS-manuel.txt", + what="char", sep="\n") Read 1550 items > textfile<-tolower(textfile) > M=NA > for(i in 1:length(textfile)){ + line=textfile[i] + M=c(M,strsplit(as.character(line),"")[[1]]) + } > T=table(M) > T M ' - \t ! " % & ( ) , . / 5308 1049 86589 44 3 3 2 2 370 391 6609 4909 12 : ; ? @ ] _ ~ ’ = « » ¬ ° 819 1178 113 1 1 4 1 39 1 108 107 823 3 … 0 1 2 3 4 5 6 7 8 9 a à 1 69 213 83 73 34 48 33 28 64 151 30559 1651 â ä b c ç d e é è ê ë f g 224 3 3562 14678 110 17713 63955 10354 1798 1000 5 4555 4911 h i î ï j k l m n ñ o º ô 4359 30851 226 47 1147 247 24792 12844 32525 6 25562 2 151 ö œ p q r s t u ù û ü v w 12 52 12696 4667 28237 37630 32945 25001 211 40 9 4787 164 x y z 1996 1222 343
Then, we can summarize in to see proportion of standard 26 letters, and we have, for Mauss,
and for Durkeim,
If we compare the two, we have almost the same proportions,
If we look at our book in English now, we have
i.e., if we compare with Mauss for instance
So we have much more E in French than in English, but still, people writing in English use a lot the E. So looking at the E should not give us any clue…. But we can see that in English, the H is as common as the L, or the C. Not in French, where L is much more frequent than the H. But on the picture, the C is more clear than the H. We can also look at the U, which is common in French, not in English… Here, on the keyboard, it is perfectly clear… so I guess people use it frequently.
So I would say that they write more in French than the write in English, on that computer.
Actually, the same idea has been used a long time ago on calculators to see that Benford’s law works: some numbers are really used (as well as the legend pretends that some pages in logarithm books were never used….), see here orthere. So Baptiste, if one day the keyboard is cleaned up, please send me another picture after a few weeks to see if things have changed….
An for those who cannot imagine how it is to work in some universities in France, just look at his blog (here). Pictures are unbelievable….Good luck Baptiste….