Quality and Impact of Academic Research
This programme aims to develop a critical assessment of the quality and impact of academics, academic research, and academic journals.
The programme grew from an early critical evaluation of received wisdom in The persistent myth of high expatriate failure rates and a later paper in Journal of Organizational Behavior, 2002, which placed the myth in the wider context of how referencing errors undermine our academic credibility. The original article has generated considerable interest and many ISI and Google Scholar citations, while the 2002 JOB article is now included in many PhD courses.
In pursuing research in this area, I have also become interested in critically assessing the impact of academic research as well as the role of editors and editorial boards in the publishing process. Recently, I have therefore started several new projects in this area.
- First, with Isabel Metz, I am looking at determinants of editorial board diversity. We have collected data for 60 journals, covering 16,000 editorial board members and nearly 10,000 articles. A first paper from this database focusing on gender diversity in editorial boards was published in the Academy of Management Learning and Education, whilst an update with 2009 data analyses the development of female editorial board memberships over time for five management fields, journals of four different ranks, and two geographic regions.
- Two papers on geographic diversity were published in Management International Review (2013) and European Journal of International Management (2012) and several articles on the role and standing of editors are in preparation.
- Second, I have developed an interest in comparisons of research output across countries. In this context, my recent article on the publication patterns of Australian academics (High Volume, Low Impact?) has generated considerable attention. A recent paper with Axèle Giroud develops this theme by applying theories from International Economics/ Business to explain how nations compete in terms of academic disciplines, and what determinants explain why countries may have a specific advantage in one discipline over another.
A separate project under this programme is a large-scale assessment of citation-based impact metrics.
- The very popular Publish or Perish software is part of this project, which has also led to a number of papers that assess the quality and differences of the citation databases provided by Google Scholar and the Thomson ISI Web of Knowledge. Papers in this area have become very popular, with the article in Ethics in Science and Environmental Politics registering nearly 100 ISI citations within 5 years of publication.
- The experience of four years of using Publish or Perish culminated in The Publish or Perish Book: Your Guide to Effective and Responsible Citation Analysis (2010). This book documents the program's many and variable uses and shows you how to get the best out of it. Citations can be used to tell stories about academics, journals and fields of research. The book is meant to help you create effective stories, but also to teach you how to be a responsible user of research metrics. Three separate shorther and focused volumes of the book were published in 2011.
- In the same context, the usability of citation metrics such as Hirsch's h-index is evaluated and applied to various academic disciplines and journals.
- A separate article with John Mingers providing a statistical analysis of my journal ranking lists extends the service already provided to researchers through my Journal Quality List.
Several short commentaries look at measuring impact for new journals and the potential arbitrary decisions in the creation of rankings, while an article-length contribution with Nancy Adler discusses the sense and nonsense of academic rankings and won the AMLE best paper award for 2009 and became an ISI highly cited paper for 2009, ranking in the top 10 most cited articles in Economics & Business for 2009.
Two papers in Scientometrics document how the Social Sciences are disadvantaged in traditional bibliometric analyses. The first paper analyses the shortcomings of ISI Web of Knowledge document categories and argues they show a profound misunderstanding of the Social Sciences. The second paper compares citation scores for Nobel prize winners in ISI and Google Scholar and argues that Google Scholar might provide a less biased comparison across disciplines than the Web of Science.
Selected publications
Online papers - Full list of publications
- Harzing, A.W. (2014) A longitudinal study of Google Scholar coverage between 2012 and 2013, in press for Scientometrics. Available online...
- Harzing, A.W.; Metz, I. (2013) Practicing what we preach: The geographic diversity of editorial boards, Management International Review, vol. 53, no. 2, 169-187. Available online...
- Harzing, A.W. (2013) A preliminary test of Google Scholar as a source for citation data: A longitudinal study of Nobel Prize winners, Scientometrics, vol. 93, no. 3, pp. 1057-1075. Available online...
- Harzing, A.W. (2013) Document categories in the ISI Web of Knowledge: Misunderstanding the Social Sciences?, Scientometrics, vol. 93, no. 1, pp. 23-34,. Available online...
- Harzing, A.W.; Metz, I. (2012) Explaining geographic diversity of editorial boards: the role of conference participation and English language skills, European Journal of International Management, vol. 6, no. 6, pp. 697-715. Available online...
- Metz, I.; Harzing, A.W. (2012) An update of gender diversity in editorial boards: A longitudinal study of management journals, Personnel Review, vol. 41, no. 3, pp. 283-300. Available online...
- Harzing, A.W. (2011) The Publish or Perish Book, Part 1: A guide to the software, Melbourne: Tarma Software Research.
- Harzing, A.W. (2011) The Publish or Perish Book, Part 2: Citation analysis for academics and administrators, Melbourne: Tarma Software Research.
- Harzing, A.W. (2011) The Publish or Perish Book, Part 3: Doing bibliometric research with Google Scholar, Melbourne: Tarma Software Research.
- Harzing, A.W.; Metz, I. (2011) Gender and geographical diversity in the editorial board of the Journal of International Business Studies, AIB Insights, vol. 11, no. 3, pp. 3-7.
- Harzing, A.W.; Metz, I. (2011) Gender and geographical diversity in the editorial board of the Journal of International Business Studies, www.harzing.com white paper.
- Harzing, A.W. (2010) The Publish or Perish Book: Your guide to effective and responsible citation analysis, Melbourne: Tarma Software Research. More about this book...
- Metz, I.; Harzing, A.W. (2010) Practicing what we preach: The geographic diversity of editorial boards, conference proceedings of The International Conference on Economics, Business Management and Marketing, 28-30 June, 2010, Paris, France.
- Metz, I.; Harzing, A.W. (2010) Gender diversity in editorial boards of management journals: An update, Proceedings of Annual London Business Research Conference, 12-14 July 2010, Imperial College, London UK.
- Harzing, A.W.; Metz, I. (2010) Geographic diversity in editorial boards: Has the world become smaller?, conference proceedings of the 11th IHRM conference, 9th-12th June 2010, Birmingham, UK.
- Harzing, A.W. (2010) Working with ISI data: Beware of Categorisation Problems, www.harzing.com white paper.
- Harzing, A.W. (2010) Citation analysis across disciplines: The Impact of different data sources and citation metrics, www.harzing.com white paper.
- Metz, I.; Harzing, A.W. (2009) Gender diversity in editorial boards of management journals, The Academy of Management Learning & Education, vol. 8, no. 4, pp. 540-557. Available online...
- Adler, N.; Harzing, A.W. (2009) When Knowledge Wins: Transcending the sense and nonsense of academic rankings, The Academy of Management Learning & Education, vol. 8, no. 1, pp. 72-95 [Winner of the 2009 AMLE Outstanding article of the year award.] Available online...
- Harzing, A.W.; Wal, R. van der (2009) A Google Scholar h-index for Journals: An alternative metric to measure journal impact in Economics & Business?, Journal of the American Society for Information Science and Technology, vol. 60, no. 1, pp 41-46. Available online...
- Harzing, A.W.; Wal, R. van der (2008) A Google Scholar h-index for Journals: A better metric to measure journal impact in Economics & Business?, paper presented at the 2008 Academy of Management annual meeting, August 8-13, 2008 - Anaheim, California. Available online...
- Harzing, A.W. (2008) Comparing the Google Scholar H-index with the ISI Journal Impact Factor, www.harzing.com white paper.
- Harzing, A.W. (2008) Arbitrary decisions in ranking studies: A commentary on Xu, Yalcinkaya, and Seggie (2008), Asia Pacific Journal of Management, vol. 25, no. 4, pp. 685-689. Available online...
- Harzing, A.W.; Wal, R. van der (2008) Google Scholar as a new source for citation analysis?, Ethics in Science and Environmental Politics, vol. 8, no. 1, pp. 62-71. Available online...
- Harzing, A.W. (2008) On becoming a high impact journal in International Business and Management, European Journal of International Management, vol. 2, no. 2, pp. 115-118. Available online...
- Harzing, A.W. (2007) Reflections on norms for the h-index and related indices, www.harzing.com white paper.
- Harzing, A.W. (2007) Google Scholar - a new data source for citation analysis, www.harzing.com white paper.
- Mingers, J.; Harzing, A.W. (2007) Ranking journals in Business and Management: A statistical analysis of the Harzing Dataset, European Journal of Information Systems, vol. 16, no. 4, pp. 303-316. Available online...
- Harzing, A.W. (2007) Reflections on the h-index, www.harzing.com white paper.
- Harzing, A.W.K; Metz, I. (2006) An analysis of the gender diversity in editorial boards: How do International Business journals compare? conference proceedings of the ANZIBA annual meeting, November 16-18, Wellington, New Zealand.
- Metz, I.; Harzing, A.W. (2006) An analysis of the gender diversity in editorial boards of Management journals, paper presented at the 66th annual meeting of the Academy of Management, Atlanta, August 11-16.
- Harzing, A.W. (2005) Australian research output in Economics & Business: High volume, low impact?, Australian Journal of Management, vol. 30, no. 2, pp. 183-200. Available online...
- Harzing, A.W. (2002) Are our referencing errors undermining our scholarship and credibility? The case of expatriate failure rates, Journal of Organizational Behavior, vol. 23, no. 1, pp. 127-148. Available online...
- Harzing, A.W. (1995) The persistent myth of high expatriate failure rates, International Journal of Human Resource Management, vol. 6, May, pp. 457-475. Available online...
