Category Archives: Opinion

Competitive NZ?

Shaun Hendy’s blog has some more comments on the theme of science, technology and innovation, prompted by our slide down the World Economic Forum rankings. His idea that scale is the main problem seems consistent with what I see in Europe – it is so easy to confer with the right people here (low spatial transaction costs!) Government policies have a huge role to play. The comparison made in the comments with Singapore is interesting, although it is very easy to run a country when you don’t have to be democratic about it. I have been talking to yet another successful Israeli – the amount of money and hard work put into science and technology there is staggering.

I think it is likely that the real problem is just culture. We don’t value investment in these things enough, owing to ignorance and different values to some extent (sport and leisure more important than intellectual pursuits). I hope things change soon – I don’t really want to leave, but the pull of the rest of the world is growing for people like me.

Recommending research articles

Challenge for AGT researchers: come up with a good mechanism to help the most noteworthy papers get the most notice!

As a start, Noam Nisan is trying an experiment to get people to recommend interesting papers in Algorithmic Game Theory/Economics. Journals no longer serve well in disseminating results quickly – is it possible to dispense with them altogether by using another way to give a signal of quality and noteworthiness?

I have doubts that this particular proposal will succeed, because of incentive compatibility problems. When we cite a paper when writing our own, it is usually taken as a recommendation (negative citations do occur, but relatively infrequently I guess). If we don’t cite it, a referee or the author may notice and our reputation will suffer. If we do cite it, our paper is probably more likely to get published. So there are some incentives to participate in that form of “recommendation”. However, recommending papers via Noam’s proposed scheme doesn’t seem to me to have as strong an incentive to participate. Who has the time to do it, if the recognition obtained for good recommendations is so low? Still, it’s good to have a first step, and I am sure with enough time and money a nice system could be devised that cuts out the for-profit journal publishers.

Ranking universities

Ranking universities according to “quality” or “impact” (mostly of research) has become a major enterprise in recent years (see Wikipedia page). There is even a blog devoted solely to the topic. Administrators at universities seem to be taking the rankings very seriously (at least when the results are favourable to them). Apparently, recent low rankings of French universities helped politicians to decide on reforms to the system.

The most famous rankings are those published by Times Higher Education and by Shanghai Jiao Tong University. THE used the company QS for its rankings until this year, when it has switched to Thomson Reuters. QS is continuing to rank.

The rankings have attracted much criticism. In particular, from what I have read the Shanghai ones seem to be not reproducible using the stated methodology, and the THE-QS relied heavily (40% weighting) on a survey of academics about reputation, which only attracted about 3500 participants. THE this time is trying for 25000.

I received an invitation to be one of the reputational survey participants, and I filled in the survey. There were many questions that required me to name (unprompted) up to 10 universities worldwide that I considered to be excellent in research (in my self-chosen area of expertise) or teaching, or would recommend students to attend (match the first few characters of the user input, with an option to write in places not in their database). The same thing was done for a geographic area with which I claimed the most familiarity. One drawback of this survey for me was that it is not possible to go back and change any answers. Once I had mistakenly selected North America to rank, I couldn’t change my mind after deciding I really knew Australasia better.

I was also asked which features of universities were most important in rankings, and given (I think – haven’t received anything) a free subscription to THE for 6 weeks for the effort. They will also send me a preview of the survey results.

Chess and Computers

Garry Kasparov’s book review is a lot more interesting than the book probably is. A lot of really interesting anecdotes from one of the greatest chessplayers ever (perhaps the greatest). I remember following his first match with Karpov via the newspapers while at high school. Now my  local paper doesn’t cover any sort of chess news, and doesn’t even have a recreational column. I must try to do something about that sometime.

Science and NZ prosperity

The NZ Institute’s discussion paper Standing on the Shoulders of Science and slides from a talk by Philip McCann (which I saw on Hard News) have come my way today. It seems that more urgency is being felt by decision-makers in NZ about the poor productivity growth of recent decades, and science R&D is being promoted more. I hope something actually happens, because these problems have been known for a long long time.

CS conferences

The debate has flared up again, and I have only a little to contribute, but here it is.

Features of the present system, some of them very different from most other disciplines:

  • intense competition for acceptance into prestigious conferences
  • a very large number of conferences overall, most rather small
  • conference papers tend to be shorter with fewer details than journal papers
  • conference papers count as much, if not more than, journal papers
  • conference papers are not refereed as deeply as journal papers, but have to be acceptable to a broader audience
  • conference proceedings to some extent set the research agenda for the community

Negatives associated with these features:

  • too much emphasis on quantity over quality
  • too much emphasis on safe, easily packaged research
  • too high a pace of publication (as opposed to genuine innovation)
  • difficulty when deciding where to publish interdisciplinary research
  • too much cost (money, environmental impact) of all the conference travel

Solutions:

  • conference papers should not count toward evaluation of an individual researcher
  • professional bodies such as ACM and IEEE should hold fewer conferences (some large and general) and emphasize their journals
  • give more credit for writing reviews, surveys and books
  • use arXiv for all new papers, and create overlay journals

I realise that implementation of these may not be easy, but it has to start somewhere. I wonder whether some people in (T)CS have become addicted to a constant round of conference travel. It really doesn’t seem a good use of society’s resources.

Conferences could retain the role of setting the agenda, selecting the best papers and maintaining a sense of community, through invited plenary lectures, special sessions in trendy areas, etc. Fewer papers would be published, and less repetition would occur. Small invited workshops could still be held, to counterbalance the large open conferences. Reviews and surveys help to set the agenda and allocate attention efficiently, as do prizes and special sessions at meetings.  This system works fine for mathematics. At least for the more mathematical parts of CS, this would work. As for the more “engineering” oriented parts, IEEE also has journals, and requires fast referee reports. It should be possible to improve the CS journal system, if the vast amount of time spent on refereeing for conferences was spent instead on refereeing for journals.

A concrete suggestion for improving scientific papers

Change editorial policy so that all articles must contain a section labelled “Our contribution” that explicitly states how the paper in question improves on previous work, and why the reader should care about it. I got this idea from an article by Simon Peyton Jones but my contribution is to add the element of compulsion! It is amazing how many papers require the referee or reader to guess what the point of writing them was.

CS exceptionalism

The long-running debate among some members of the (theoretical) computer science community about its publishing habits has heated up again, after Lance Fortnow’s column to appear in CACM arguing for a reform of the competitive conference system, Noam Nisan’s “opposing” view and many comments on both blogs. Central to the debate seems to be the concern on one side that CS is out of step with practice in other disciplines, while the other side seem to believe that through luck or design, the younger field of CS has hit upon a superior model to other disciplines.

CS is a strange field, a mixture of mathematics and engineering, crudely speaking. There has been ongoing debate over whether CS is a science or part of engineering, and the two camps in the conference argument seem influenced by those who think CS is a science and those who think it is isn’t. I certainly belong to the first, although sometimes my faith is shaken. But surely CS will eventually be recognized as a mature science with substantial links with other disciplines. These should lead to its exceptional publishing culture changing almost automatically.

More on Iran situation

After several weeks of following the news obsessively, I need to get back to work. I thought some summary information might be helpful for others. Of course these sites are all in English. My Farsi reading level is like that of a primary school child, and Google Translate’s Persian alpha-version is not really adequate, so I am sticking to English mostly.

Background: general Iranian history; a nice diagram of the power structure in Iran; some key players; more description of Mousavi, Khamanei, Ahmadinejad; coup leaders;

Analysis: articles on politics by Mohammad Sahimi at TehranBureau; Juan Cole; Ali Ansari; statistical analysis of the election results by Walter Mebane and FiveThirtyEight.com;

News updates: Huffington Post; National Iranian-American Council blog; Windows on Iran;

Advocacy: Abdorahman Boroumand Foundation; ICHRI; Amnesty International; Haystack