This section gives the background to university rankings and discusses 4 rankings important to Australian universities:
Various national rankings of universities have been published since the magazine U.S. News began its annual rankings of US colleges and universities in 1983. The Princeton Review, a company based in New York city known for its test preparation courses, education services and books first published its annual Best colleges ranking of ‘the best 357 colleges’ in 1992. The weekly Canadian magazine Macleans first published its ranking of Canadian universities in 1991. TheCenter for measuring university performance, a US research body, started publishing its annual ranking of the top US research universities in 2000.
The Deutsche Akademische Austauschdienst (German academic exchange service) and the German weekly news magazine Stern has published the Centrum für Hochschulentwicklung (Centre for higher education development) ranking of Germany’s 250 universities since 1998. The magazine Asiaweek published is report Asia’s best universities from 1997 to 2000. The magazine ceased publication in 2001. The UK’s Sunday Times first published its rank of UK universities in 2001. Ross Williams and Nina Van Dyke of the University of Melbourne’s institute of applied economic and social research published their International standing of Australian universities in 2004.
The first important international ranking of universities was not published until 2002 when the Swiss Federal Government’s Zentrum für Wissenschafts und Technologiestudien(Centre for Science and Technology Studies) published its Champions league of research institutions which ranked universities and other research institutions by their number and impact of research journal publications during 1994-1999. The centre’s current Champions league is for publications in 1998-2002. Shanghai Jiao Tong University’s institute of higher education first published its academic ranking of world universities in 2003. The Times Higher Education Supplement published its first World university rankings in 2004.
The Swiss Federal Government’s Zentrum für Wissenschafts und Technologiestudien (Centre for Science and Technology Studies) Champions league ranks research institutions by their performance in research journal publications. The centre ranks the top 683 institutions by 4 measures:
The centre got its data from 3 citations indices weighted thus:
| arts and humanities citation index | 5% |
| social sciences citations index | 11% |
| science citation index | 84% |
The centre counts research publications in journals only, and therefore does not count books and other research publications. This understates research publications in the arts, humanities and social sciences which publishes more books than the empirical fields. The centre also gives research journal publications in the arts, humanities and the social sciences less weight than their share of all research journal publications.
Shanghai Jiao Tong University’s institute of higher education’s world academic ranking of universities ranks universities by their share of 6 factors weighted thus:
| alumni who won a Nobel prize or a Field medal for mathematics | 10% |
| staff who won a Nobel prize or a Field medal for mathematics | 20% |
| highly cited researchers in 21 broad subject categories | 20% |
| articles published in Nature and Science | 20% |
| articles in Science Citation Index-expanded and Social Science Citation Index | 20% |
| correction for institutional size | 10% |
| Total | 100% |
|---|
The Nobel prizes, Field medals, Nature and Science articles and Science Citation Index-expanded and Social Science Citation Index are all heavily weighted towards the sciences, and as the Times (2004: 2) points out, 2 research publications measures overlap.
The Times Higher Education Supplement’s World university rankings is based on 5 factors with these weights:
| survey of 1,300 academics in 88 countries | 50% |
| % overseas staff | 5% |
| % overseas students | 5% |
| staff:student ratio | 20% |
| citations per staff | 20% |
| Total | 100% |
|---|
The Times Higher Education Supplement says that it asked academics to name the top institutions in the areas and subjects on which they felt able to make an informed judgement. The Times doesn’t say what ‘top’ might be, but traditional research performance was probably the most important factor. The Times says it ‘balanced’ its survey by academic discipline and location. The Times gave 5% for the proportion of academic staff recruited from overseas. Since this isn’t systematically reported for Australian universities they were all allocated a score of 49 out of 100. The Times gave 5% of its score for proportion of international students. While this may be a good indicator of the quality of universities in other countries, it is not a good indicator of the quality of Australian universities.
Williams andamp; Van Dyke’s International standing of Australian universities is based on 19 factors weighted thus.
| Factor | weight |
|---|---|
| Views of vice chancellors of overseas highly ranked universities and Australian deans | 8% |
| Revenue per student | 11% |
| Undergraduate students (25%) | |
| Median tertiary entrance score | 11% |
| Retention rate | 3.5% |
| Staff:student ratio | 3.5% |
| Overall satisfaction in course experience questionnaire | 3.5% |
| % of bachelor graduates proceeding to a higher degree | 3.5% |
| Postgraduate students (16%) | |
| PhD completions | 5.6% |
| All postgraduate progression rate | 4.8% |
| Overall satisfaction in postgraduate experience questionnaire | 5.6% |
| Research (40%) | |
| All research publications | 4% |
| Publications in the Essential Science Indicators (ESI) for laboratory disciplines | 4% |
| ESI publications for economics and business and social sciences disciplines | 2% |
| ESI citations for laboratory disciplines | 6.8% |
| ESI citations for non laboratory disciplines | 3.2% |
| Membership of an Australian academy | 8% |
| Number of staff who were included in the ISI-ESI list of highly cited authors | 2% |
| National competitive research grant income | 6% |
| Other research income | 4% |
| Total | 100% |
Williams andamp; Van Dyk asked vice chancellors of 172 overseas highly ranked universities to compare the international standing of each Australian university with the standing of universities in their continent and they asked Australian deans to compare the international standing of each Australian university with US universities. Williams andamp; Van Dyk’s respondents placed most emphasis on the ‘quality/international standing of staff’, which is based on research performance.
Williams andamp; Van Dyk’s calculations of revenue per student are wrong. They include the dual sector universities’ funding for vocational education and training and divide that by higher education students only, thus ranking these institutions misleadingly high on this measure. When this and other corrections are made this factor will be mainly another measure of research revenue. The scale for undergraduate students is too compressed for about the top 60% of institutions on this measure and so does not discriminate much between most institutions. Williams andamp; Van Dyk’s measure of all research publications overlaps with their measures of publications in the Essential Science Indicators (ESI), and Williams andamp; Van Dyk’s ESI publications and citations measures weight laboratory disciplines at twice that of the other disciplines.
Equity or diversity should be added to Williams andamp; Van Dyk’s measure of undergraduate intake. Extensive quantitative data are readily available on institutional student equity which, incidentally, are of far better quality than the institutional data on student entry scores and discriminate well between institutions. While these measures are meant to describe what institutions’ international standing are actually based on rather than what they should be based on, they nevertheless have a normative aspect and in any case diversity of student body is at least rhetorically important as an indicator of the quality of the undergraduate experience in the US and is increasingly important in the UK. Equity of undergraduate intake should be expressed as a separate factor and not submerged within other factors, however small a weight it may be given.