Australian research output in Economics & Business: quantity over quality?

This paper – written during my first sabbatical leave at the University of Melbourne in 2004 – marked the start of my interest in research evaluation and research impact. Since then, and especially since the launch of the Publish or Perish software in 2006, I have developed a substantive research programme in this area.

ranking

Australia ranked 4th ... or 19th

Using the Thomson Reuters Essential Science Indicators – which ranks the top 1% most cited universities world-wide in each field – I noticed that Australian research output (i.e. the number of published papers) in Economics and Business was very high for the size of its population. Australia ranked 4th ex-aequo with Germany, being outranked only by the USA, the UK and Canada, all (much) bigger economies. Australia in turn outranked much bigger European economies such as France, Italy, and Spain.

However, the story for impact (citations per paper) was quite different; Australia ranked only 19th in the world on this measure and was outranked not just by the USA, the UK, and Canada, but also by the aforementioned Southern European countries, as well as most small European countries such the Nordic countries, Austria, Switzerland, the Netherlands, and Belgium.

Interestingly, it turns out that this mismatch in ranking for productivity and impact was not a temporary fluke. When I verified Australia’s rankings for Economics and Business in the Essential Science Indicators ranking in September 2016, i.e. more than twelve years after the original data collection, I found them to be identical, i.e. 4th for the number of papers and 19th for citations per paper. So it seems the phenomenon is persistent. New Zealand outranks Australia at #16, a ranking that is higher than its ranking for publications (#24).

Looking at individual universities paints this picture in even starker relief. As the table below shows, the top-3 universities in Economics & Business in Australia have further improved their world ranking in terms of the number of papers. However, their ranking in terms of citations per paper has slipped to the lowest regions of the Essential Science Indicators (there are currently 270 universities ranked for Economics & Business).

 

Rank no of papers

Rank citations/papers

University 

2004

2016

2004

2016

Australian National University

48

46

148

262

University of Melbourne

59

34

149

229

University of New South Wales

67

51

141

251

Are research incentives driving behaviour?

Why would Australia display this particular pattern? Although it is difficult to find any conclusive evidence, my 2005 paper offered a range of conjectures. For many I provided a comparison with the UK and the Netherlands, the latter being similar in terms of population size. My money is on #6 as the most important reason.

  1. High student/staff ratios: Government funding for universities has declined steadily in Australia; Australian universities generate a large part of their income from (international) student fees. This means they spend a lot of time and resources on recruitment and that student/staff ratios are generally very high. This is especially true in Business & Economics as this is the most popular choice for international students. At the University of Melbourne, student/staff ratios in this field were nearly five times as high as in Medicine and more than twice as high as in Engineering, leaving academics in Business & Economics less time to do high-impact research. My teaching load in Australia was certainly higher than it ever was in the UK and the Netherlands and my classes much bigger.
  2. Less flexible salary structures: Australian universities are at a disadvantage to US universities in terms of the salaries they can offer. US universities can generally offer substantially higher salaries in order to attract top researchers from all over the world. However, this does not explain why Australia fares worse than for instance the UK and the Netherlands, neither of which are known for their generous academic salaries. Moreover, in the last decade Australian salaries have become more attractive in comparison to other countries and salary structures more flexible.
  3. Research tied to Australian context: Australian research might be tied to the Australian context and hence be less interesting to international researchers. Articles published in top journals are indeed less likely to have an Australian focus. However, this does not explain why articles that are published are cited less. Of course this might indicate that North American researchers, who account for the bulk of citations in the field, are biased against Australian researchers. However, this does not really explain why Australia fares so much worse than for instance the UK and the Netherlands.
  4. Business schools have a shorter history: Business education in Australia has a shorter history than in North America and the UK. However, while business schools might generally have been established much later in Australia than in the other countries, faculties of Economics & Commerce have a far longer history. And again this does not explain why Dutch academics fare better.
  5. Australian research might simply reflect local managerial practice: Local Australian research might simply reflect local managerial practice. It is generally recognised in Australia that Australian managers have never been at the forefront of innovations in management. If Australian management lags behind its international counterparts, it might be more difficult for Australian academics to do innovative research to produce high quality articles.
  6. Australian research evaluation rewards quantity over quality: In contrast to for instance the UK, the Netherlands and New Zealand, the Australian academic climate seems to reward quantity over quality. Yearly governmental research evaluations require the submission of all publications rather than just a sub-set. A journal article receives 1 point regardless of whether it is published in the field’s top journal or a third-tier journal. It is unclear why this policy would influence Economics & Business more so than other disciplines, although the discipline’s focus on instrumental rationality, efficiency & money might lead academics in this discipline to respond more strongly to incentives than e.g. Egyptologists. In addition, the pressured existence of academics in Economics & Business might lead them to go for the alternative that appears to maximise return on time invested in terms of publication points and promotion: publication in journals that are ISI listed, but have relatively low quality standards and high acceptance rates. Especially since the second half of the 1990s, Australia seems to lag behind the UK and the Netherlands in terms of publications in the top quartile of ISI rankings. The start of this period coincides with the introduction of quantity-based research evaluation in Australia.
  7. Lack of research funding in Business & Management: I would normally expect competitively funded research to provide higher impact research output. A comparison of Australian Research Council Discovery grants for 2002–2005 showed that whereas funding in Economics was commensurate with its proportion of academic staff, Business & Management only accounted for 2.1% of the grants even though 12.6% of academic staff in Australian universities was employed in this field. In comparison, Engineering & Technology received 14.5% of the grants awarded for 6.9% of staff.

Conclusion

Australia’s lack of research impact – measured as citations per paper – in Economics & Business appears to be a persistent phenomenon. Unfortunately, many of the potential explanations refer to circumstances that are difficult to change. However, a change in research evaluation to a system that rewards quality as well as quantity might resolve some of these problems and can be implemented at the level of individual universities. In 2003 Linda Butler contrasted the differential research policies of the University of Queensland (focusing on quality) and the University of Western Australia (focusing on quantity). In 2016 UQ was the highest ranked Australian university in terms of citations per paper and UWA was one of the three lowest ranked. Food for thought…

References

  • Butler, Linda (2003). Explaining Australia’s increased share of ISI publications—the effects of a funding formula based on publication counts. Research Policy, 32(1): 143-155.

How to cite this blog?

A free pre-publication version of this paper is hyperlinked. If you’d like to have an official reprint, just drop me an email. If you refer to my work I would prefer you to cite the original paper. However, if you want to refer to the new information in this blog, please do so as follows:

  • Harzing, A.W. (2017) Australian research output in Economics & Business: quantity over quality?, Harzing.com blog, http://www.harzing.com/blog/2017/05/australian-research-output-in-economics-business-quantity-over-quality.