Mon Jun 24, 2013

Citation Networks in Philosophy: Some Followup

June 26th, 2013. I’ve corrected some errors in the dataset. They changes don’t effect the substance of the post. All of these involve merging variant citations to the same work. Notable changes in the graph are the increased prominence of Davidson (1980), van Inwagen (1990), Putnam (1975), and (to a lesser degree), Wittgenstein (1953). I thank Brad Wray for drawing my attention to some of these errors.

Last week’s posts on A Co-Citation Network for Philosophy and Lewis and the Women generated a fair amount of discussion amongst philosophers, and I also got quite a few emails. I want to thank the philosophers who emailed me corrections or spotted duplicates in the item list. I will continue to fix minor errors along these lines. A few people asked whether it would be possible to have a poster-like presentation of the data. Here it is as a PDF file suitable for printing. There is a PNG version as well.

Substantively, I got various requests and suggestions about various ways the data might be further sliced and diced. The two most common responses were, first, for more journals to be added to the list of sources, e.g. leading specialist journals such as Ethics or the British Journal for the Philosophy of Science. And second, people also asked for more detail on what has been happening in the recent past, for instance by focusing on the last ten or even just five years. In response to the first question, and before saying something about the second, it may be worth saying a little bit more about what motivated the original choice of journals and time-frame.

Prestige and Generality

Academic disciplines are complex, dynamic fields where papers, projects, and people compete for attention. There are various research literatures devoted to getting a descriptive sense of what is happening in various fields over time, and—much harder—explaining why things turn out the way they are. Fields are nested and interconnected. Different measures of prominence or quality capture different aspects of the careers of ideas, papers, and people. Prestige orderings for different entities (articles, books, people, journals, presses, departments, subfields, disciplines, universities) are more or less formalized, and are also connected.

In principle all of is supposed to be happening on the basis of quality. At least, when it comes to their own successes, most people want to claim quality is the ultimate explanation. Few really believe it, except perhaps people who have always and only been successful. When it comes to their own failures, reasons other than poor quality immediately suggest themselves. Many of these reasons are quite plausible. From the point of view of any particular career, as Max Weber remarked, “Academic life is a mad hazard”. On the other hand, we also see quite a lot of structure, and competent actors within particular fields typically have a strong practical sense (and often a working theory) of how this structure works and who it favors.

My focus in the Philosophy case was on what’s been happening in high-prestige, generalist journals over the past twenty years. In the long run, what counts as a prestige outlet is up for grabs. In the short run, it’s fixed. Prestigious outlets claim to speak for the field. If they wish to keep doing so, they have to be attentive to where their field is going. Otherwise, the action will move somewhere else, as the field redefines itself and leaves the journal behind. (This is a real possibility in Philosophy, as the volume and specialization of work seems to have been steadily increasing since the 1980s.) I chose four high-impact, high-prestige, nominally generalist journals to work from. The data take all the articles published in these journals between 1993 and 2013 and analyze every item cited in them—about 34,000 citations altogether. The idea is to show what work people in “the field” were talking about (and jointly talking about) over the past twenty years. We pick nominally generalist rather than specialist journals because they’re the most prestigious overall, and they claim to represent the best, most central work in the field.

Work from various subfields in philosophy is not equally represented in these journals. Some is represented in only limited or attenuated ways. Other topics are completely absent. But attempting to define both quality and scope is part of what generalist journals do in academic disciplines. It is presented as the application of judgment, but to those who lose out it can just as easily be read as the exercise of power. It’s important to note that I don’t take a position on this here, I just take it as empirically given. All that matters for our purposes is that these journals are taken to be high-prestige general-interest outlets in the field of English-speaking professional philosophy. I am not judging the quality of subfields or topics. I cannot possibly address here how, for example, pre-career and so-called “pipeline” processes lead women to choose one topic rather than another, or exit one area and enter another.

Rather, I just take it as given that if nominally generalist journals are publishing mostly, say, Philosophy of Mind, Epistemology, and the Philosophy of Language, and those journals are in practice seen as high-prestige generalist outlets—and not abandoned by high-status members of a discipline that has moved on, for example—then what doesn’t get published in these outlets is, in practice, not as central to the field considered from a sociological point of view. The journals were selected to fix a working definition of the high-status end of the field. The 2,200 articles published in them between 1993 and 2013 have passed a stiff test in getting published at all. The citations they contain tell us what was being talked about in these high-prestige outlets. The intrinsic worth of this disciplinary conversation is not something I pretend to judge. But I don’t need to.

Citations are Cheap and Important

Next, why citations? I look at citations because—unlike publishing in these journals, and also unlike getting a job at a top philosophy department—citations are cost-free. An author can in principle cite whomever they take to be a relevant or important interlocutor from the entire history of philosophy, or from any department or journal in the world. The only constraint is the judgment of authors (publishing in these elite places) about whose work is worth talking about. The pattern of these judgments, as revealed in co-citation practices, is what we are interested in. This choice is not really free, of course. In the background is the weight of expectations (from referees, editors, or one’s peers) about who and what to cite. But this is part of how judgments of quality and value are collectively and unequally produced inside academic fields. As I said in the original posts, Philosophy is a particularly interesting case because absolute citation rates are low. Choosing to cite someone carries more weight here than in other fields, and—I argue—makes the co-citation graph a relatively good measure of where the conversational action is in the discipline.

In general, scholarly reputations are made first by being read and talked about informally, catching the attention of influential actors and being judged promising by them. Next comes publications in a relatively high-profile outlet, then by citation in ongoing published conversations. At each stage of this process, we have the opportunity for familiar issues to crop up. For example, perhaps because it is a relatively small and homogeneous field based in large part on excellence in argument rather than quality of data, pure promise—manifested first interpersonally—seems to go further in philosophy than other disciplines. In comparison to most other humanities and social science fields, it is possible (though by no means typical) to get quite far in one’s career on the basis of very little published work. Ultimately, though, published work is necessary, and citations to it grow and consolidate one’s reputation. A general feature of highly cited books or articles is that they gain even more citations as the Matthew Effect takes hold. Other work is ignored, and either moves to a discussion at a more peripheral location or is forgotten.

Are there any Women to Cite?

With the work of Kripke and Lewis dominating the center of the graph, several people asked whether it was all that surprising to see so few women in the dataset, given that there seem to have been so few prominent women working in these dominant areas over the past forty or fifty years. In retrospect, here again it looks like there simply wasn’t anyone to cite. I can’t resolve this question definitively here. My personal prior here is that enough women have worked in these areas for us to have expected at least a few more from the entire history of the field. You may think that all and only the best work has been cited, that all of it is by men, and that’s the end of the matter. But while I am no modal realist, I think the actual configuration of citations in my data is not the only one we might have ended up with.

An alternative hypothesis to pure quality always winning out is that work by women is not cited in high-prestige outlets. Further, this is linked to a cumulative process that reduces the likelihood that talented women will keep working in fields where their work is not discussed. This is not all that controversial an idea in the general literature on citation, although—in part because of negative feedback and endogeneity problems—it is hard to demonstrate cleanly. For the data I have to hand, these issues also get me uncomfortably close to naming and discussing work by people I know personally. This is partly why I was reluctant to narrow the focus to the past five years or the last decade. Narrowing the window for any period makes the data noisier. It becomes impossible to see what’s happening. Coming closer to the present means we focus on people who are working in the field right now. I do not like dealing with a noisy dataset in the context of a touchy subject like whether one’s success is “deserved” in some absolute sense. However, as I’ve repeatedly been asked whether things look different in the past decade, I will make some brief points.

Patterns since 2000

Rather than slicing the data into two ten-year segments and looking at the top 500 cites in each one—something that would make things messy at the lower end of the citation list—let’s take our twenty-year most-cited list and examine articles published in the 2000s alone. This means we are looking at hot papers or books: recently-published items, with relatively little time to accumulate a high citation count, that nevertheless do well enough to make it on to the twenty-year Top 500 list. Such items can of course be authored by very well-established philosophers, or—more interestingly for our purposes—they might be written by younger people making a big early-career impact. Here is the complete list of highly-cited items from the 2000s:

Rank    Cites    Item    Typically Cited In
4 83 Williamson T 2000 Knowledge Limits Nous, Philosophical Review
30 42 Sider T 2001 4 Dimensionalism Ont Nous, Journal of Philosophy
41 36 Hawthorne J 2004 Knowledge I Ottenes Nous, Philosophical Review
82 27 Tye M 2000 Consciousness Colro Nous, Philosophical Review
86 26 Stanley J 2005 Knowledge Practical Nous, Philosophical Review
86 26 Pryor J 2000 Nous Nous, Philosophical Review
165 19 Lewis D 2000 J Philos Journal of Philosophy, Philosophical Review
179 18 Campbell J 2002 Reference Consciousn Nous, Mind
179 18 Stanley J 2000 Mind Lang Nous, Mind
195 17 Schiffer S 2003 Things We Mean Nous, Mind
211 16 Woodward J 2003 Making Things Happen Philosophical Review, Journal of Philosophy
233 15 Bennett J 2003 Philos Guide Conditi Philosophical Review, Nous
233 15 Derose K 2002 Philos Rev Philosophical Review, Nous
233 15 Fantl J 2002 Philos Rev Nous, Philosophical Review
233 15 Noe A 2004 Action Perception Philosophical Review, Nous
262 14 Dancy J 2000 Practical Reality Nous, Mind
262 14 Gibbard A 2003 Thinking Live Nous, Philosophical Review
262 14 Stanley J 2001 J Philos Journal of Philosophy, Mind
262 14 Martin M 2002 Mind Lang Nous, Mind
262 14 Martin M 2004 Philos Stud Nous, Mind
262 14 Hitchcock C 2001 J Philos Philosophical Review, Journal of Philosophy
288 13 Peacocke C 2004 Realm Reason Nous, Mind
288 13 Byrne A 2003 Behav Brain Sci Philosophical Review, Nous
288 13 Byrne A 2001 Philos Rev Nous, Mind
324 12 Stanley J 2000 Linguist Philos Nous, Philosophical Review
324 12 Sosa E 2007 Virtue Epistemology Nous, Mind
324 12 Baker L 2000 Persons Bodies Const Philosophical Review, Mind
324 12 Hajek A 2003 Synthese Philosophical Review, Mind
324 12 Travis C 2004 Mind Nous, Philosophical Review
324 12 Merricks T 2001 Objects Persons Philosophical Review, Nous
324 12 Pereboom D 2001 Living Free Will Philosophical Review, Journal of Philosophy
363 11 Cappelen H 2005 Insensitive Semantic Nous, Philosophical Review
363 11 Dowe P 2000 Phys Causation Philosophical Review, Journal of Philosophy
363 11 Chomsky N 2000 New Horizons Study L Nous, Mind
363 11 Williamson T 2007 Philos Philos Mind, Philosophical Review
363 11 Rysiew P 2001 Nous Philosophical Review, Nous
435 10 Soames S 2002 Rigidity Unfinished Philosophical Review, Nous
435 10 Arntzenius F 2003 J Philos Philosophical Review, Journal of Philosophy
435 10 Velleman J 2000 Possibility Practica Philosophical Review, Nous
435 10 Paul L 2000 J Philos Philosophical Review, Journal of Philosophy
435 10 Cohen S 2002 Philos Phenomen Res Philosophical Review, Nous
435 10 Lange M 2000 Natural Laws Sci Pra Mind, Nous
435 10 O’connor T 2000 Persons Causes Metap Philosophical Review, Nous
435 10 Huemer M 2001 Skepticism Veil Perc Nous, Mind
435 10 Foot P 2001 Natural Goodness Mind, Nous
435 10 Chalmers D 2001 Philos Rev Nous, Philosophical Review
435 10 Conee E 2004 Evidentialism Nous, Journal of Philosophy
435 10 Wright C 2002 Philos Phenomen Res Philosophical Review, Nous
435 10 Heck R 2000 Philos Rev Philosophical Review, Nous

There are 49 articles altogether. Several of the authors are relatively young. Here, “Young” means they got their Ph.Ds in the 1990s or later. Working down the list, these younger people include: Sider, Hawthorne, Stanley (several times), Pryor, DeRose, Fantl, Noe, Hitchcock, Hajek, Cappelen, Merricks, Dowe, Rysiew, O’Connor, Paul, and Heck. (I may have missed one or two: please email me corrections.) There is one woman amongst the lot. The only other item published in the 2000s and written by a woman is by an established philosopher, Lynne Rudder Baker. Some of this recent work by relatively younger men—John Hawthorne (Ph.D ‘91), Ted Sider (Ph.D ‘93), Jason Stanley (Ph.D ‘95), Keith DeRose (Ph.D ‘90) and Jim Pryor (Ph.D ‘97)—even manages to crack the twenty-year Top 100, and in three cases the Top 50. That’s pretty impressive. Notably, the most-cited work by a 90s-cohort Ph.D is not on this list, because it was published in 1996. This is The Conscious Mind by David Chalmers (Ph.D ‘93).

By contrast, we have to go down to joint 435th to find a well-cited paper authored in the 2000s by a—in fact, the only—woman from a comparable Ph.D cohort. This is by Paul (Ph.D ‘99). One of the reasons this is a little awkward is that I am married to this particular data point. I hope I don’t need to emphasize that my wife has had no involvement in the data collection or analysis presented in any of these posts. Incidentally, I didn’t expect there to be so few women in the Top 500. I thought the prevalence of women would be poor, but not terrible.

If we look further down the dataset to the next 500 most-cited items, there are quite a few items written by women from the 1990s-era Ph.D cohort that all of the highly-cited younger men come from. One clear example, brought to my attention by Brian Weatherson, is the case of Delia Graff Fara (Ph.D ‘97). A prominent and very well-respected philosopher, Graff Fara is a full Professor at Princeton working in one of the general areas regularly published in our four journals. She is an obvious case of a very successful woman with a Ph.D from the 1990s who—to Brian’s and many people’s surprise—failed to have an item in the top 500. She is not the only such case. It’s not that her work isn’t cited. She has one paper in the dataset with eight citations, and another with seven. It just isn’t cited enough.

She isn’t alone. Just below the threshold in the data are items by senior women philosophers such as Jennifer Hornsby, Margaret Gilbert, Rae Langton, Karen Neander, Elizabeth Fricker, and Robyn Carston, together with more work by Ruth Millikan, Christine Korsgaard, Linda Zagzebski, Judith Jarvis Thomson, Nancy Cartwright, Susan Haack, and the linguist Angelika Kratzer. Restricting ourselves to items by women with Ph.Ds from the 1990s or later, we find the lower 500 contains work by Nomy Arpaly (Ph.D ‘98), Tamar Gendler (Ph.D ‘96), Miranda Fricker (D.Phil ‘96), Rosanna Keefe (Ph.D ‘97), Helen Beebee (Ph.D ‘96), and Karen Bennett (Ph.D ‘00).

This pattern of results cannot demonstrate, but in my view does suggest, the hypothesis that the process at work here—even in recent years—is not one where women cannot be cited because they simply don’t exist in the relevant subfields favored by journals. Rather, women publish, yet their work is not cited. Citation is not the only measure of success. One can be highly placed without necessarily being highly cited. And, of course, by definition not everyone can be a citation star. But given how easy it is to cite people, this only makes the gender disparity all the more stark. It is very surprising to see only male philosophers from the ‘90s cohort so well-represented in the Top 500, and even the Top 100. The 1990s were not the 1950s. And yet essentially none of the women from this cohort are cited in the conversation with anything close to the same frequency, despite working in comparable areas, publishing in comparable venues, and even in many cases having jobs at comparable departments. It is one thing to see such a lopsided pattern in faculty placements, where getting a job is usually a zero-sum competition, and the stakes are very high. To see it in reference lists is worrying, given that compared to the cost of a tenured line, citations are very cheap indeed.