On multicore Stata MP performance

Since I am in the middle of some very interesting discussions and experiments concerning this topic: it really does look like the difference in speed between the different multi-core/multi-processor Stata/MP versions is considerable. Right now I am performing a comparison exercise* with my friend and co-author Miguel, where, it turns out, a number of certain estimations with a 8-core Stata/MP 11 take about 10 almost 15 times less real time (I know, I hardly believe it myself) than (almost) the same estimations done with a 2-core Stata/MP 11. Taking the median across all estimations commands in Stata, an 8-core will outperform a 2-core by a factor of 2.28 (NB: this ratio appears to be larger than the price ratio of these two multicore versions!): I computed this figure from other stats available in this 250 pages report on Stata MP's performance.

The next quest should be to assess Stata's bold claim: "From dual-core laptops to the big iron of multiprocessor servers, Stata gets the most out of multicore systems. No other statistical software comes close" (my emphasis in bold). If true, that surely ought to boost Stata's status in the statistics/econometrics community (part of which-- I plead guilty too-- is currently infatuated with using other programming languages like Ox, Fortran, Gauss, Matlab etc, while typically leaving Stata for simple exercises or data manipulation).


*brief update footnote here, for clarification (well, at least part of the clarification..., the huge, factor 10 15, execution time differential remains somewhat puzzling): we cannot really perform this comparative exercise directly, as "(almost) the same" here means  that we do have  identical specifications (and the same software and operating system), but nonetheless different  (very large) panel datasets-- and in this particular case the difference in the structure (specifically, the "connectedness" of the cross-sectional-time series data, but the intention was not to give too many details) of those datasets matters, beyond an eventual difference in  the number of observations (for our purpose virtually the same); if we ran identical specifications on exactly the same dataset, with the 8-core and respectively the dual-core Stata/MP, the speed ratio could not be higher than 8/2=4, the theoretical limit. Given the type of exercise performed here, the real time differential due solely to the difference in core numbers is most likely close to this limit.

Sunday night econlinks

• Cheaptalk post on the theory and history of open source software (with link in their text to the very interesting new NBER paper by Schwarz and Takhteyev)

• "A neat project would be to infer the extent of school spirit from the estimate of the price elasticity implied by the prices that different university cemeteries charge"-- very much so, indeed-- though looks only applicable/relevant for a few places in the USA (most likely unrepresentative even of the USA entire university universe, not to even mention Europe or others). Nevertheless, very selective sample aside, this is an interesting idea: read Dan Hamermesh's entire post on Freakonomics.

• Niall Ferguson concise on Quis custodiet ipsos custodes? and the whole craze to re-regulate finance..., via Alex Tabarrok on MR.

• Buffett and Munger seem to like it finance-simple, though they surprinsingly appear to entirely blame the scientists & the science rather than the practioners who learnt and applied the science wrongly, especially so when this science is typically very clear about its assumptions (but could be the journalist's fault here, e.g., by taking things out of the context... hopefully). See also a related earlier article in Wired.

• The Economist's gloomy snapshot of Central and Eastern Europe facing the crisis. Not over yet, but so far not as bad as it could have been...

Chess games, more and less serious

First, the powerful Corus Chess tournament has started today (first game tomorrow) in Wijk aan Zee. Time for last year's Group A winner, Magnus Carlsen, to really make history. Ivanchuk did quite well lately, but I still don't see him with too many chances against the Viking Chessmaster, this time. Aronian is, however, always 'unpredictable': hence, if anything, he's the one Magnus should watch out for the most... Group B will also be very interesting: my bet is on the new hope, Fabiano Caruana: I saw some unbelievably smart games he played in different tournaments last year.

Second, computer chess games for the less professional players, such as myself :-). In order to have an idea whether I should get the new chess program for my Ipod (see also here), I should ask Susan Polgar how does the tChess compare to the Kasparov Chessmate , the latter available within the set of games offered on the screens in KLM intercontinental flights (now you know that other than watching good movies--2nd bullet point-- I also play a lot of chess on such flights...). I am obviously no pro when it comes to chess, but I won 15 games (and lost only 3) against the highest level the Kasparov Chessgame has ('strength' 2300, ie. playing against Kasparov's avatar...), which clearly says more about that game than about myself :-). If the Ipod chessgame is similar or lower in strength, I wouldn't bother (though in any case, I think it is absolutely great to have such a chess game on Ipods and Iphones)... All in all, I am definitely looking forward for Fritz-like strength in chess games on such devices as Ipods, KLM screens and the like...

Econlinks for 04 Oct '07

• Try a free "publish or perish" software. Though it does pretty much the same (and only that, as far as I had time to test...) as scholar.google.com

• Rules vs. discretion: Posner continues Becker's entry on rules versus discretion in the context of central banks' decisions, but by doing so he also provides an excellent general discussion on the law and economics of the subject. I don't think anybody can beat Posner in that area.

• Vote (ex post I wouldn't bother...) for the free market hall of fame. Could have been done much better, the way they did this is a shame, really. And some of the results so far reinforce that...

• A September '07 profile of Robert Barro, yet another (potential) physicist turned (macro)economist :-).

Econometrics: A few reasons to use Ox over Gauss

...though I am definitely going to try Gauss as well and learn it better than I know it at the moment (and I certainly think one should try many programming languages; each of them could have particular comparative advantages in specific routines etc.). Below I place some excerpts from some very interesting discussions on an ox discussion list that I consult quite frequently. I will leave the names of the authors out since everybody can trace the fragments on the list linked above (more precisely, we are talking about the archive for April, so here) and keep only what I consider the essential excerpts (I will also place links to the full messages at the end of the respective fragments ). They pretty much contain what I would have to say over the subject as well (obviously I am not so experienced as most of the researchers who answer here; as a personal note, I particularly like the references over the power of the BFGS log-likelihood maximization routine- that came clear, inter alia, in part of the research for a paper I co-authored, and of course the easy extension using C/C++, programming language which I learned far before I ever got to learn/do econometrics- ok, so I might be somewhat biased when preferring an object-oriented, modular, econometrics software :-)).

So, we start here, with the question:

"Dear Ox Users, I am a PhD student who tries to choose between Gauss and Ox. I have tried to ask some people which one to choose, I have read some documents to guide myself in chossing between the two, but still could not make myself clear on which one to choose. [...]" Link.

And some answers I particularly liked were:

"I think that trying to find "the best econometric software" is the wrong way to think about statistical/econometric programming nowadays. 15/20 years ago, people used to learn only one or two econometric software ,but I think that the things are very different now. Even if Gauss highly used in finance, both Gauss and Ox are great software, but sometimes Gauss is better and sometimes Ox is better. I simply depends on what you do. [...] If you are new to econometrics programming, I would say that Ox is easier to learn than Gauss, it is very powerfull (especially if you do arfima, markov models and panels) and it's free for universities." Link.

"[...]I think Ox has an excellent syntax based on C (with its extensions the most used programming language, there must be a reason). I think it is easy to read and to learn. Ox, if you want it, is also object oriented, and it is very easy to make reusable code or packages to share with others. There is not so much documentation on Ox as for Gauss for two reasons: 1) Gauss is still more widespread, 2) the official documentation of Ox is Excellent and public (on the web). Graphs in Ox just look better and are very easy to integrate into LaTeX documents. Ox is fast, but also Gauss is pretty quick. Ox may be extended in C. It is easy to make simple GUIs with Ox." Link.

"[...] 1) Gauss got there first, and is American. Hence it has a huge user base, and lots of code written for it. Alas, it is a poorly designed language. It's too unstructured, and it's easy to write incomprehensible spaghetti code. 2) Ox got there second, and is British. It has, inevitably, a more modest user base. It is, however, a beautifully designed language, combining all the power of matrix programming with the logical structure of C. (It is actually a lot simpler to write than C.) If you follow the recommended coding conventions (naming, indenting) then it is easy to write elegant, compact and self-documenting programs. It's got object-orientation and classes if you are into that, but you don't need to be. So - no contest frankly. If you want to learn a new language for your research, you will not regret starting with Ox. Try Gauss later when you are ready for the worst." Link.

"[...]I used Gauss for econometric programming in the past, and I have to say I did not like it. I oftentimes ended up with"incomprehensible spaghetti code". For starters, the language is not case sensitive, which bothers me. Ox is great. Its syntax is similar to C, it's fast, it has an excellent numerical library, and so on. Just try to do some Monte Carlo where you numerically maximize a log-likelihood function using BFGS in Gauss and Ox, and you will notice the difference. Additionally, please note that you can run Gauss code using Ox. Finally, Ox is free (for academic research and teaching), which is a huge plus in places like Brazil, where we don't have access to university wide site licenses. " Link.

PS. It should be said that, to be completely fair, one should also listen to the viewpoints of the people on a Gauss-users list :-). If anybody has such references, I'd be happy to link them here.

Gapminder, now on Google tools as well

I once blogged (among other things) about this extremely interesting and welcome software, Gapminder, developed by Hans Rosling and his team from the Karolinska Institute in Sweden. It seems that the software is now included as part of the google tools, which makes it even more popular and easier to disseminate everywhere. Recently Greg Mankiw also spotted how useful this software is. Try it out: inter alia you can get a lot of nice graphs of several economic indicators (and visualize their dynamics over time, as well) for Romania.

30 essential pieces of free software for Windows

Via The Simple Dollar. I was already using (albeit not all the time) 12 of them. You're the best if you can guess which ones :-).

Blog backup

Am citit acest articol din SmartFinancial, dar gasesc ca se prezinta problema, se scriu paragrafe si paragrafe fara cap si fara coada si - at the very end of the day- nu se ofera solutii simple si la indemana oricui (ceva tipic in jurnalismul nostru). Ori prima solutie la care oricine ar trebui sa se gandeasca in materie de securitate a blog-ului este backup-ul blog-ului si - pasul imediat urmator si esential- cum sa faci treaba asta in modul cel mai comod/eficient posibil.

Cateva randuri deci pe topicul backup-ului. Evident poti sa faci backup-uri in multe feluri si pana la urma depinde si de preferintele personale. Intr-o enumerare aleatorie si incompleta: daca tii blog-ul pe serverul propriu poti sa faci backup cum vrei si cand vrei si poti sa activezi un proces de backup-ing automat- urmatoarele idei sunt deci pentru situatia cand nu ai blog-ul pe serverul propriu; in plus unii provideri de blogspace ar putea sa iti ofere direct optiunea de backup (sau, echivalent, sa iti dea indicatii exacte cum se poate face asta: blogger.com merge de exemplu pe ideea asta , neavand optiune explicita pentru backup), deci am terminat si cu optiunea asta- in continuare trecem doar pe linia: ai blog-ul gazduit de unul dintre provideri si nu ai o optiune de backup explicita sau implicita, deci vrei variante 'universale'. Ce faci in cazul asta? Ei bine, ai o gramada de posibilitati si aici (unele evidente...): poti pur si simplu sa ai toate textele (sau imaginile etc) postate, salvate anterior pe orice suport digital; o alta posibilitate este sa activezi functia de trimis orice entry-uri catre o adresa email si sa le pastrezi pe toate acolo; similar alternativelor anterioare se poate sa salvezi pur si simplu fiecare post, fie in format .html, fie in format text etc. , dupa dorinta. Important insa este ca template-ul blog-ului sa fie salvat din timp, de fiecare data, indiferent care alternativa de backup de mai sus este folosita ( daca tii la formatul template al blog-ului si nu doar la continut).

Personal prefer o alta alternativa pe care nu cred sa o stie toata lumea. Exista acum diverse produse software care pot face un backup complet oricarui site, fie ca e vorba de un blog, fie ca e vorba de altceva. Si functioneaza de minune (am testat un asemenea produs atat pentru backup-ul weblog -ului cat si cel website-ului meu). Un astfel de software pe care il recomand oricui (Windows users...) este HTtrack Website Copier. Procesul backup poate dura mai mult daca site-ul respectiv are o dimensiune mare, dar pentru un blog e relativ rapid. Pentru cei care folosesc Mac-uri o alternativa la soft-ul de mai sus este Webgrabber.