While we’re running our Crowdsourced Sociology Rankings, people have been looking a little more closely at the U.S. News and World Report rankings. Over at Scatterplot, Neal Caren points out that U.S. News’s methods page has some details on the survey sample size and response rates. They’re bad:

Surveys were conducted in fall 2012 by Ipsos Public Affairs … Questionnaires were sent to department heads and directors of graduate studies (or, alternatively, a senior faculty member who teaches graduate students) at schools that had granted a total of five or more doctorates in each discipline during the five-year period from 2005 through 2009, as indicated by the 2010 “Survey of Earned Doctorates.” … The surveys asked about Ph.D. programs in criminology (response rate: 90 percent), economics (25 percent), English (21 percent), history (19 percent), political science (30 percent), psychology (16 percent), and sociology (31 percent). … The number of schools surveyed in fall 2012 were: economics—132, English—156, history—151, political science—119, psychology—246, and sociology—117. In fall 2008, 36 schools were surveyed for criminology.

So, following Neal, this tells us the Sociology rankings are based on a survey of 117 Heads and Directors with a response rate of 31 percent, which is thirty six people in total. For Economics you have 33 people, for History 29 people, for Political Science 36 people, for Psychology 40 people, and for English 33 people. The methods page also notes that they calculate the scores using a trimmed mean, so they throw out two observations each time (the highest and the lowest). The upshot is that the average score of a department is likely to have rather wide confidence intervals.

Update: These numbers are too low. Read on.

I guess it’s possible that U.S. News might mean that the effective N of, e.g., the Sociology survey is 117, and that’s the result of a larger initial survey which yielded a 31 percent response rate. On that interpretation they initially contacted 378 departments (or thereabouts). That would be a non-standard way of describing what you did. Normally, if you give a raw number for the sample size and tell us the response rate, the raw number is the N you began with, not the N you ended up with. More importantly, there aren’t 378 Ph.D granting Sociology departments in the U.S.—a quick check of the Survey of Earned Doctorates suggests that there were 167 in 2010. That suggests that 117 is about right for the number who had awarded five or more in the past five years, and that this is the total initial N. Same goes for Economics, which has 179 Ph.D programs in the 2010 SED.

Then again, the wording in the methods can also be read as saying every department received two surveys (“Questionnaires were sent to department heads and directors of graduate studies … at schools that had granted a total of five or more doctorates … during the five-year period from 2005 through 2009”). Looking more closely at the available SED data for 2006 to 2010 (one year off the USNWR dates, unfortunately), I found that 115 Sociology Departments met the stated criteria of having awarded five our more doctorates in the previous five years. If both the Dept Head and DGS in all those departments got a survey, this makes for an initial maximum N of 230. But that is still quite far from the 378 or so needed, if 117 is supposed to mean the 31 percent who responded rather than the total number initially surveyed.

So it seems like the most plausible interpretation is that for Sociology the number of schools surveyed is in fact 117, that every school received two copies of the questionnaire (one to the Head, one to the DGS or equivalent), but that the 31 percent response rate means “percent of schools from which at least one response was received”, and so the total N surveys for Sociology is somewhere between 36 and 72 people, with a similar range of between 30 and 80 for the other departments.

As many of you are by now aware, U.S. News and World Report released the 2013 Edition of its Sociology Rankings this week. I find rankings fascinating, not least because of what you might call the “legitimacy ratchet” they implement. Winners insist rankings are absurd but point to their high placing on the list. Here’s a nice example of that from the University of Michigan. The message here is, “We’re not really playing, but of course if we were we’d be winning.” Losers, meanwhile, either remain silent (thus implicitly accepting their fate) or complain about the methods used, and leave themselves open to accusations of sour grapes or bad faith. They are constantly tempted to reject the enterprise and insist they should’ve been ranked higher, and so end up sounding like the apocryphal Borscht Belt couple complaining that the food here is terrible and the portions are tiny as well.

The best thing to do is to implement your own system, and do it better, if only to introduce confusion by way of additional measures. Omar Lizardo and Jessica Collett have already pointed out that U.S. News decided to cook the rankings by averaging the results from this year’s survey with the previous two rounds. They provide an estimate of what the de-averaged results probably looked like. Back in 20011, Steve Vaisey and I ran a poll using Matt Salganik’s excellent All Our Ideas website, which creates rankings from multiple pairwise comparisons. It’s easy to run and generates rankings with high face validity in a way that’s quicker, more fun, and much, much cheaper than the alternatives. So, we’re doing it again this year. Here is OrgTheory/AAI 2013 Sociology Department Ranking Survey. Go and vote! Chicago people will be happy to hear can vote as often as you like. So, participate in your own quantitative domination and get voting. For the duration of the survey, you can even do it right here with this handy widget:

Note: This post is by L.A. Paul and Kieran Healy. The paper it draws on is available here as a PDF.

You should think carefully about whether to have kids. It’s a distinctively modern decision. Until comparatively recently, producing an heir, supplying household labor, insuring against destitution, or being fruitful and multiplying was what having a child was about. Nowadays the decision to bear a child is freighted with a more personal significance—assuming you are physically able to do so, and lucky enough to be well-off and well-situated. Children are an enormous responsibility, we are told, and you should be sure you really want to have one before you go ahead and do it. In particular, you’re supposed to reflect carefully on what it would be like. You weigh the options and make a decision.

Crucially, this involves assessments of your future experiences. You imagine your life with and without kids, and think about what it would be like or feel like to have that experience. In the language of philosophers, you must think about the phenomenology of the experience. When it comes to children, people argue endlessly about what you ought to do. Some claim motherhood is a supremely fulfilling vocation. Some wearily raise their hands (after wiping off spit-up milk) and beg to differ. Others see liberation in the decision to avoid parenthood. They complain about the presumptions of a culture that equates child-rearing with happiness or self-realization, or that looks with pity or suspicion on the indecently happy and child-free. Insofar as there is any detente in the Mommy Wars, though, it’s around the idea that you should personally reflect with great care on these issues and decide for yourself whether this … this—what? Grand adventure? Prison sentence?—this experience is for you.

That sounds like a reasonable compromise, until you realize no-one knows what it’s like to have a child, until they have one. It’s a phenomenologically transformative experience. Fear not, veterans of the Mommy Wars. We are not saying it’s wonderful, or that people who don’t experience it have somehow failed at life. We just mean that people are very different afterwards, in ways they cannot anticipate. The evidence for this is everywhere. New parents laugh ruefully at their detailed pre-kid plans to fit “the baby” into their existing lives. “You ruined everything/In the nicest way”, as songwriter Jonathan Coulton says. Those who choose to remain child-free, meanwhile, bridle at the insulting suggestion that they are missing out on something. Yet they see their friends get body-snatched one at a time, cocooned in minivans and unable to stay out past eight, lost to civilized life, unrecognizable. Something happened to them. Even the parent who reacts to their new situation with numb disbelief, or shock and depression, has a transformative experience. These reactions have their own cruel character because they break so sharply with the official story.

Crying baby courtesy of photosavvy.

Stripped of judgmental overtones, the transformative character of becoming a parent is not a controversial idea. The trouble is, transformative experiences throw a wrench in decision-making based on future experiences. In theory, a rational choice is a series of steps: first determine the possible outcomes, and the costs and benefits associated with each one; then assign a probability to each outcome to calculate its value; finally, choose the option that gives the highest expected value. Real decisions are rarely so clean cut, because we are imperfect calculators and it is probably impossible to figure expected values with precision anyway. Yet this is the decision-making standard we aspire to. For it to work, you must at least be able to assess the costs and benefits of the most important outcomes.

But in this case, the most important outcomes include things like “what the experience will be like for me” or “what it will be like to be a parent”. If becoming a parent is a transformative experience, you can’t know in advance what it will be like for you. You can’t assess the costs and benefits of these outcomes, since you can’t know their values—and so if you choose based on what you think it will be like for you, you can’t even approximate a rational decision-making procedure. Our ordinary understanding of the choice to have a family or remain childless—all that careful weighing of options based on what it’s going to be like for you in the future—is based on a fantasy. You don’t know what it is going to be like. So you can’t rationally make the choice by weighing options involving the experience of parenthood.

You probably have some objections. You might say, “What if I decide to have a child solely because I want to pass along some DNA?” Or, “What if I decide to remain child-free solely because there are too many people on this earth already?” That’s fine. If you’re really not basing your decision at all on what being a parent is going to be like for you then you can make a rational decision. But relying only on criteria like that is not the usual way to decide to have kids.

You might say, can’t a rational decisionmaker adopt a different decision rule, one specially designed to deal with difficult choices? She can—but at a price. For example, consider a play-it-safe rule that says, “Simply choose the option whose worst case scenario is the best one relative to every other option’s worst case scenario.” This rule could help you choose between options without incorporating any special knowledge about what it would be like for you to have a child. Sounds reasonable, but it leads to strange results when considered from any particular individual’s point of view. Take Suzy, for instance, who believes she’d love to have a baby. If the best worst-case scenario involves not having a child (if this is better than having a child and bitterly regretting it, say), then if Suzy follows the play-it-safe rule, she should stay child-free, regardless of her own feelings—as should everyone else following the rule.

What about testimony from people like yourself? Can’t you look at them and rationally expect to have a similar experience if you make a similar choice? No. Without just the sort of self-knowledge you’d get from your own experience of having a child, you can’t know how the experience will affect you, and so you can’t know whether you’re more like the parent or the child-free person. As the saying goes, you get experience just after you need it. Even worse, parental testimony is unreliable. A parent may claim she is happier now than she would have been if she had not had her baby, but that may be because she cannot truly imagine life without it. Once a person has had a child, it becomes psychologically very difficult for her to assess what it would be like if she’d never had it. So even after having the child, she probably can’t weigh the different outcomes.

What about making a simple bet? Shouldn’t you just play the odds and choose to have a family, setting aside what you personally think it will be like? Isn’t it just obvious that having a child will make you happier? The standard account of choosing to be a parent certainly reinforces this view, with its endless talk of deep fulfillment. But the evidence suggests that’s nonsense. The highs may be higher for parents, but the lows are lower. Measures of overall personal happiness suggest that parents with children at home are less happy than those without children. Moreover, individuals who have never had children report similar levels of life satisfaction as individuals with grown children who have left home. If you merely want to play the odds, one shouldn’t have a child. But does that mean you shouldn’t have a child? No! You might be one of the people who find the experience of parenthood fulfilling.

We are not arguing that it is right or wrong to have a child. Nor are we saying people shouldn’t be happy with their choice. You can be happy with a child or blissfully child-free. But if you are happy, you shouldn’t congratulate yourself on your wise decision—you should be thankful for your good luck. Choosing to have a child involves a leap of faith, not a carefully calibrated rational choice. When surprising results surface about the dissatisfaction many parents experience, telling yourself that you knew it wouldn’t be that way for you is simply a rationalization. The same is true if you tell yourself you know you’re happier not being a parent. The standard story of parenthood says it’s a deeply fulfilling event that is like nothing else you’ve ever experienced, and that you should carefully weigh what it will be like before choosing to do it. But in reality you can’t have it both ways.

I recently made some updates to the Emacs Social Science Starter Kit. I maintain the SSSK for my own convenience, but other people have found it useful as well. By now there are a lot of little bits and pieces in the kit, so I thought it might be useful to do a listicle highlighting some of the conveniences it offers. As a reminder, the motivation behind the kit was to allow researchers, faculty, and grad students working in the social sciences to get started with Emacs. The general principles the kit tries to facilitate are discussed in my (now slightly outdated) article on Choosing Your Workflow Applications.

Rather than try to cover all the many great features of every package the kit installs—all the products of a terrific amount of work by other people, and some of which, like AucTeX and ESS, are very large and sophisticated—I am just going to mention some of the useful things you might immediately use once you’ve installed the starter kit.

1. Ido-mode and Smex mean you don’t have to Memorize a Huge Battery of Shortcuts Like any powerful text editor, time-saving keyboard shortcuts are central to using Emacs efficiently. While it’s easy to grasp the general logic of Emacs key-chords, getting specific ones into your muscle memory takes longer, especially for features which you might not use all that often. In the interim, the SSSK sets up ido-mode and Smex, and together they allow you to search buffers, file names, and functions quickly and flexibly. They also remember what you’ve searched for before, so they are more likely to prompt you with the right option the next time. So, for example, if you want to browse the kill-ring (Emacs’s multi-item clipboard) but you don’t remember the keyboard shortcut, starting to type M-x browse-kill-ring will complete the command very quickly. If you can’t remember the command to insert multiple cursors in a region (see below), starting to type M-x mc/mark will bring up all the commands that match, and you can use C-s to cycle through the options. Thanks to ido-mode, search functionality for finding files, open buffers, functions, and other commands is very smart and flexible. If you partly remember the name you’re looking for, or type a shortened or partial version of it, it will likely be found, and it gets better the more you use it.

2. Use M-x without the Meta key Most Emacs commands are accesed either via C-x or M-x. As is standard, Meta is mapped to the Mac keyboard’s “Option” key. But you can also invoke M-x by doing C-x m, which means most things can get done using either C-x or C-x m. It’s a little more convenient.

3. Search inside multiple files with rgrep or ack You can search for text inside files in your current directory (and recursively all directories underneath it) in one of two ways. C-x C-r invokes rgrep, which will ask you for a search term and a file type. If you have Ack installed, you can do C-x C-a to search all files in your working directory for some term. You can also do M-x ack-same to search only in files of the same type as the current buffer.

4. Autocomplete text as you write Autocomplete mode is useful for completing functions in the scripting or programming languages it knows about (including R), and it can also be useful in long text documents. It’s turned off by default for most text modes (except org-mode) in the SSSK, but you can enable it in a buffer with M-x auto-complete-mode.

5. Quickly Resize, Rotate, and Cycle through Windows Early on when using Emacs you learn to split a frame into two or more windows with C-x 2 or C-x 3. As you work (e.g., with R code in the left-side buffer and an R session running in the right-side buffer), you often want to move the cursor to a different window, move buffers from one window to another, or cycle back and forth between different window configurations you’ve used. The starter kit lets you move the cursor between windows using Shift and the arrow keys. C-c m “rotates” windows, e.g. by moving the left window to the right side and vice versa. C-c-<up> and C-c-<down> cycle through the window configurations you’ve created in the past. Shift-C-<left>, Shift-C-<right>, Shift-C-<up>, and Shift-C-<down> resize split windows a little bit at a time.

6. Make Shift-Enter do a lot in ESS When working with R and ESS, if you open an R file in the main buffer then htting Shift-Enter vertically splits the window and starts R in the right-side buffer. If R is running and a region is highlighted, shift-enter sends the region over to R to be evaluated. If R is running and no region is highlighted, Shift-Enter sends the current line over to R. Repeatedly hitting Shift-Enter in an R file steps through each line (sending it to R), skipping commented lines. The cursor is also moved down to the bottom of the R buffer after each evaluation.

7. Turn on Highline-mode when you need it Some people find it easier to edit code when they can see exactly where the cursor is on the screen. M-x highline-mode adds a highlight to the current line, so you know where you are.

8. LaTeX Symbols display nicely in the buffer The kit includes latex-pretty-symbols. When editing a text file, you can type TeX commands like \sigma and \gamma and they will display properly (as unicode symbols) right in the buffer. The LaTeX command is still there underneath.

9. Multiple Cursors One of the flagship features of Sublime Text 2, editing with multiple cursors can be a really useful dynamic alternative to search-and-replace when you want to make the same changes to a bunch of lines in a file. Thanks to Magnar Sveen, multiple cursor support is now available in Emacs and the SSSK installs the package. The easiest way to see it in action is to select a region and do M-x mc/edit-beginning-of-lines or select a region and try M-x mc/mark-all-in-region to search for a bit of text to replace on each line.

10. Minimal Mode When working full screen, I like to make scroll bars and the dividing lines between windows more unobtrusive. M-x minimal-mode (or C-c s for short) toggles them on and off.

There’s a lot more in the SSSK, from handy functions to a few themes (try M-x load-theme solarized-dark or zenburn) and you can browse the notes in the kit’s org files to learn more about what it sets up for you, and how to customize it to your own needs.

I’ve made some updates to the Emacs Starter Kit for the Social Sciences. In addition to various bits of cleanup, bug fixes, and package updates, I eliminated the need for any git submodules. This simplifies the installation, and allows for people to install the starter kit from a zipfile instead of via git (although git is still recommended). Because major components like Auctex and ESS are now available as packages, less has to be contained in the kit itself.

The main problem I encountered while testing the new version is a certain flakiness on the part of Emacs when it contacts the ELPA and Marmalade servers to download the various packages. I’ve encountered three different errors: a long period of hanging without response; a generic “can’t parse HTTP buffer” message; and a specific insistence that “The package ‘auctex’ is not available for installation”. The latter one in particular is strange, because it’s definitely there in the ELPA package repository.

Dealing with these errors is annoyingly close to magical handwaving. The first option is to simply quit and relaunch Emacs and have it try again. This often works (though more than one try is sometimes required). The second, specifically for the auctex error, is to do M-x list-packages and then retry via quitting and relaunching Emacs. Sometimes the installation goes smoothly first time, sometimes this kind of messing around is required before Emacs finally decides auctex is in fact available and installs it. If these problems persist after a couple of tries, the best thing to do is simply to install the offending package or packages manually. Do M-x list-packages and in the resulting buffer search or scroll down the list to, e.g., auctex, mark it for installation by pressing i and then install it (or them) by hitting x. With the packages in place, restart Emacs and the starter kit will finish setting itself up.

Unfortunately I don’t think there’s anything else I can do to make these intermittent installation errors go away. If you use the kit, let me know if you run into any other problems with the installation.