Reflections on norms for the h-index and related indices
Prof. Anne-Wil Harzing, University of Melbourne
Web: www.harzing.com
Email: pop@harzing.com
© Copyright 2007-2010 Anne-Wil Harzing. All rights reserved.
Seventh version,
26 March 2013
Introduction
Ever since Publish or Perish has been available, academics have asked me for "norm scores" for the various indices. I guess everyone likes to compare himself or herself to see how they are doing. I have always hesitated to do this as these scores are so easily taken out of context and can take on a life of their own.
However, most publications discussing the h-index and related indices deal with academics in the (Natural) Sciences. There are large differences in research output between the different disciplines (see Reflections on the h-index and Citation Analysis Across Disciplines). H-indices in the Natural Sciences are much higher than in the Social Sciences and Humanities. Hence, I felt that in order to support academics in the Social Sciences and Humanities it would be appropriate to provide some systematic evidence that lower citation indices can be expected in these disciplines.
A calculation of citation metrics for an individual academic requires one to be at least familiar with the field in question (in order to be able to eliminate publications by authors with similar names) and preferably with the individual’s work. Therefore, my "norm scores" only pertain to the two fields that I am familiar with: Management and International Business.
Method
It is difficult to establish an appropriate comparison group. However, I think most academics will agree that fellow academics who have been elected as presidents of the peak professional organization in their field would probably constitute an appropriate (if ambitious) benchmark. Therefore, I have calculated the various citation metrics (h-index, g-index, contemporary h-index and individual h-index) for presidents of the Academy of Management (AoM) and the Academy of International Business (AIB) for the last 25 years. Since AIB presidentship does not rotate every year, the data are based on 15 individuals, whereas for AoM they are based on 25 individuals.
It is important to note that the record of specific individuals might be both underestimated and overestimated. Most of the publications of academics that were presidents in the early years will be relatively old. Google Scholar does not perform as well for older publications, because these publications and the publications that cite them have not (yet) been posted on the web. Hence their citation metrics might be understated. On the other hand, given that Google Scholar cannot go back in time, we calculated their citation metrics as of the current date, rather than the date at which they became president, hence overstating their citation metrics.
Since most presidents of AIB and AoM are senior, well-established scholars, they will have published a lot of their important work a while ago. This means that their contemporary h-indices might be lower than those of newly-minted professors. On the other hand, the overall top scorer in the combined group of AIB and AoM presidents (John Dunning) is one of the oldest academics in the group and has a high contemporary h-index.
Results
With the aid of Publish or Perish I found the following citation metrics as of 6 March 2007:
| As of 6 March 2007 (mean/median) | h-index | g-index | hc | hI,norm |
|---|---|---|---|---|
| Academy of Management Presidents | 20 / 18 | 43 / 41 | 11 / 10 | 15 / 14 |
| Academy of International Business Presidents | 15 / 13 | 31 / 23 | 9 / 8 | 13 / 10 |
I repeated this exercise on 26 March 2013, i.e. 6 years later. As can easily be verified, all citation metrics have gone up significantly. This is caused partly caused by a natural increase of citation levels for this group of high-performing academics. However, it also likely to reflect a further expansion of Google Scholar coverage over the last six years. Hence care is required when comparing citation metrics over time. This is true for all citation databases, but probably more so for Google Scholar than for ISI Web of Science and Scopus.
| As of 26 March 2013 (mean/median) | h-index | g-index | hc | hI,norm | citations/year |
|---|---|---|---|---|---|
| Academy of Management Presidents | 40 / 35 | 104 / 84 | 24 / 20 | 31 / 27 | 341 / 190 |
| Academy of International Business Presidents | 30 / 24 | 71 / 55 | 16 / 11 | 23 / 22 | 161 / 75 |
It should be noted that AoM and AIB presidents are a select group of high-performing academics. Hence one cannot expect every professor in Management or International Business to display similar citation metrics. However, those professors that do display similar citation metrics (or even exceed them) could certainly be seen as high performers themselves.
Other norm scores
I invite academics to submit norm scores for similar groups in their own subdiscipline with a short description of their search parameters. I would suggest that comparison groups need to include at least 10 academics (and preferably more) to avoid idiosyncrasies. If I am confident that the data are accurate, I will post them on my website with attribution to the person who has compiled them. Please submit your results to pop@harzing.com.
- Research Performance of Senior Level Marketing Academics in the Australian Universities: An Exploratory Study Based on Citation Analysis, a paper by Mohammed A. Razzaque and Ian F. Wilkinson, presented at the Australia New Zealand Marketing Academy Conference (ANZMAC), University of Otago, New Zealand, Dec 1-3 2007.
- Citation Benchmarks for Articles Published by Australian Marketing Academics, a paper by Geoff Soutar, In M. Thyne, K. R. Deans and J. Gnoth (eds.), Conference Proceedings of the 2007 ANZMAC Conference. Dunedin: Department of Marketing, University of Otago, 3515-3520.
- Measuring the research contribution of management academics using the Hirsch-index by John Mingers, published in Journal of the Operational Research Society, applies the h-index to three groups of management scholars: BAM fellows, INFORMS fellows and members of COPIOR.
- Cumulative and career-stage citation impact of social-personality programs and their members by Brian Nosek and co-authors provides benchmarks for evaluating impact across the career span in psychology, and other disciplines with similar citation patterns. In press for Personality and Social Psychology Bulletin. Supplementary page with career-stage impact calculators: http://projectimplicit.net/nosek/papers/citations/
- In Characterizing author citation ratings of herpetologists using Harzing’s Publish or Perish Malcolm L. McCallum analyzed a random sample of herpetologists. He used linear regression to analyze the influence of career length and publication count on their h-score, g-score, e-score, and m-quotient and provides mean scores for each author metric for herpetologists at various career lengths.
- Evaluating the Productivity of Social Work Scholars Using the h-Index by Jeffrey Lacasse, David Hodge & Kristen Bean, published in Research in Social Work Practice introduces the h-index and related statistics to social work faculty. It also presents the results of a comprehensive study of 337 tenure-track faculty in top-10 universities and 215 editorial board members in the field.
