Ranking universities according to “quality” or “impact” (mostly of research) has become a major enterprise in recent years (see Wikipedia page). There is even a blog devoted solely to the topic. Administrators at universities seem to be taking the rankings very seriously (at least when the results are favourable to them). Apparently, recent low rankings of French universities helped politicians to decide on reforms to the system.

The most famous rankings are those published by Times Higher Education and by Shanghai Jiao Tong University. THE used the company QS for its rankings until this year, when it has switched to Thomson Reuters. QS is continuing to rank.

The rankings have attracted much criticism. In particular, from what I have read the Shanghai ones seem to be not reproducible using the stated methodology, and the THE-QS relied heavily (40% weighting) on a survey of academics about reputation, which only attracted about 3500 participants. THE this time is trying for 25000.

I received an invitation to be one of the reputational survey participants, and I filled in the survey. There were many questions that required me to name (unprompted) up to 10 universities worldwide that I considered to be excellent in research (in my self-chosen area of expertise) or teaching, or would recommend students to attend (match the first few characters of the user input, with an option to write in places not in their database). The same thing was done for a geographic area with which I claimed the most familiarity. One drawback of this survey for me was that it is not possible to go back and change any answers. Once I had mistakenly selected North America to rank, I couldn’t change my mind after deciding I really knew Australasia better.

I was also asked which features of universities were most important in rankings, and given (I think – haven’t received anything) a free subscription to THE for 6 weeks for the effort. They will also send me a preview of the survey results.