The recent boycott of the Handelsblatt (HB) ranking for our fellow business economists (BWLer) by some of these fellow business economists has caused some stir in the econ/business blogosphere. Here, here and here are good debates on the topic.

It may be instructive to describe a little bit how top departments in the US make tenure decisions and decisions on outside senior hires. While I do not have direct experience on the former, I do have it on the latter, as the Economics Department at the University of Michigan, my former employer, involves junior people fully in the outside senior hiring process, so I have read quite a few outside letters for senior positions. Moreover,  you can gather from dinner and beer talks how things are done at other departments.

One thing is for certain: journal rankings have NEVER been part of the conversation (there is one exception in that within the top 10 departments one or typically more than one publication in the so-called top 5 is necessary, if by far not sufficient, for tenure or an outside senior appointment). In contrast, the conversation was very much about the content of the papers, their novelty, the embodied creativity, their citations, their more general influence (“Is this paper already part of a second year PhD field course?”), their believability, their exhibited care, etc. Senior search committees (and then later the faculty meeting) spend an enormous amount of time with considerable attention to the details of the papers when deliberating a candidate. A big part of the conversation is also fit – does this person fit into our intellectual environment, will he or she enrich and broaden it, do we want to work with this person? The last thing on anybody’s mind is a journal ranking by some journalist. 

Now, in Germany, allegedly – at least according to its critiques – the HB ranking plays a major role in new professorial hiring decisions. I actually doubt that and would contend that in the good places (you all know them, so no need to keep repeating them here) the U.S. model is much more prevalent. I certainly could not have cared less about the HB ranking in the one Berufungskommission that I have chaired in Germany, and yet we put three of the most promising young empirical macroeconomists somehow related to Germany on the short list (not just in my opinion).

Nevertheless, I find it believable that in lesser places the HB ranking plays a bigger role. The reason is three structural differences in German Econ academia compared to the U.S. (I surmise this is not much different in business). 1) Smallness of departments, which often leads to the situation that nobody in the field to be hired is there and can competently judge the papers of the candidates – the HB ranking is an easy way out; 2) lack of a general education in a PhD program, which means that faculty with a traditional German PhD eduaction often lack the background to talk about papers in other fields (of course, I would always defer to my theory or econometrics colleagues to judge such papers, but it’s also not true that I can be completely bamboozled in these areas); 3) I think the most important one is the lack of a department structure to begin with, which has two direct consequences: a) I honestly think that many German professors do not care as much as US professors about who is going to be a colleague – as I said, in the US senior hiring is one of the most important services one can provide to the department; and, rightfully so, as they run a chair, which is ultimately an isolated unit for which they are responsible on their own. In other words, in a chair system the benefits of having a say on new hires is much less valuable than in a department system, which is why German professors rationally do not want to exert the effort, so the HB ranking comes in handy (of course, smallness goes in the same direction, as it is rare that a potential collaborator is hired, because all fields need to be somehow covered for teaching reasons – Düsseldorf, which has recently focused heavily on IO, is a great counterexample); b) in a department system the opinion of junior professors who are often at the technological frontier of their field are much more likely to be heard when it comes to senior hiring than in a chair system, especially in fields where the department does not have many senior people. Needless to say, of course, that none of these reasons is an excuse to rely on the HB ranking – all it would take is to sample competent opinions on candidates from other departments, domestic or abroad (another thing that is regularly done in the pre hiring season in the US – people call up their buddies, talk during conferences about potential candiates, people willing to move, etc.)

So, this establishes that the leading Economics and Business departments do not use the HB ranking for their hiring decisions (the same is true, btw, for grants, etc.), so we can already see from a best practice kind of argument that the HB ranking is probably overplayed.

But there are also more substantive reasons why this is so. First, should not hiring decisions, even for senior appointments, be based on EXPECTED publications (or more generally research output)? Unless you have a system like in the UK which directly rewards buying CVs – you want somebody because you expect him to be a great colleague, great teacher, a great researcher that will bring attention to your institution (hiring nobel prize winners maybe the exception, but, hey, it is also an extreme example). By construction, the HB ranking (or any ranking for that matter) captures only the past. Now, past research output and future research output, both in terms of quality and quantity, may be correlated, but by how much? Do we know that? There are clear cases, where the correlation is low – very young and very old researchers. But there are also more subtle cases, which have to do with fit. Would this guy really be as or even more productive in our intellectual environment or maybe less, because the fit is not good? Another case has to do with research fads – what if somebody has a lot of human capital and top publications in a dying area (how many top departments would be willing to hire a specialist in mathematical general equilibrium theory or large-scale linear simultaneous equation models, even with a high HB ranking).

Another case where the HB ranking obfuscates and only the CV and the reading of the papers helps: what if somebody has accumulated a lot of HB points by publishing a lot in third-tier journals, with unoriginal ideas (say, replicating with German data AER-published results with US data), or by beating – in terms of ideas – always the same horse, just in different packaging. This happens! Or what if somebody has accumulated a lot of HB points by publishing articles in one research strand that is currently hot, but nothing else (I am assuming here that the research strand was not founded by this person), etc.

The bottom line: the HB ranking can be overplayed and should never be a substitute for careful reading of papers. Again, I actually do not believe it is, at least not as much as its critiques make believe (especially not at the good German Econ departments). In any event – downplaying it a little more in the future is probably healthy.

Nevertheless, for better or for worse, this ranking did something with German Economics and this something is for the most part good. German economists are no longer the laughing stock of the international scientific community. And for this Olaf Storbeck should be thanked. Because – the way things were done in the old days, well, let’s just say, it was not the good ol’ days. I expressly disagree with some of the comments in the Wirtschaftsphilosoph’s blog on the topic which expressed the quaint feeling that things were not so bad in the old days. They were. And the fact that the old system produced somebody like C.C. von Weizsäcker (the example given there) is really the exception, not the rule. In fact, he himself described and bemoaned the hideous backdoor deals and the monarchs of the old system at a panel discussion in Frankfurt, we both once participated in.