Wed Aug 26, 2009

The Impact Factor's Matthew Effect

Via Cosma, comes the following article:

Since the publication of Robert K. Merton’s theory of cumulative advantage in science (Matthew Effect), several empirical studies have tried to measure its presence at the level of papers, individual researchers, institutions or countries. However, these studies seldom control for the intrinsic “quality” of papers or of researchers—”better” (however defined) papers or researchers could receive higher citation rates because they are indeed of better quality. Using an original method for controlling the intrinsic value of papers—identical duplicate papers published in different journals with different impact factors—this paper shows that the journal in which papers are published have a strong influence on their citation rates, as duplicate papers published in high impact journals obtain, on average, twice as much citations as their identical counterparts published in journals with lower impact factors. The intrinsic value of a paper is thus not the only reason a given paper gets cited or not; there is a specific Matthew effect attached to journals and this gives to paper published there an added value over and above their intrinsic quality.

The full paper has some more detail. Duplicates are defined as those papers published in different journals but which nevertheless have the same title, the same first author, and the same number of cited references. With this definition the authors find 4,532 pairs of duplicates in the Web of Science database across the sciences and social sciences. (This is a pretty striking finding in itself.) Remember that the impact factor of a journal is meant to be a (weighted) product of the number of citations to articles in that journal — i.e., a journal’s prestige is a function of the quality of the articles appearing in it. But here we see that, for the same papers, the impact factor of the journal affects the citation rate of the paper. The mechanism is straightforward, but it’s neat to see it shown this way.

(Appropriately enough, I have posted this at both Crooked Timber and OrgTheory. We’ll see which one gets the links and comments.)